Touch Designer Experimental + Vulkan

2021.38110

First experimental build release for 2021.30000 series is out.

Video Overview by Elburz from The Interactive & Immersive HQ

Build 2021.38110 - Dec 17, 2021

Major Architectural Updates

Vulkan API

TouchDesigner's graphics API is now Vulkan, openGL has been retired. The port to Vulkan is part of our current efforts to modernize TouchDesigner's engine and enables the introduction of new features that are available through modern APIs such as Vulkan and Metal.

macOS uses MoltenVK, a portability project allowing Vulkan to be used on top of Apple's Metal framework ensuring we are developing with the latest API that Apple is focusing their resources on.

Vulkan can significantly reduce driver overhead compared to openGL.
Compute Shaders fully supported on all platforms including macOS now.
Future exciting features will be possible with TouchDesigner's new Vulkan architecture:

Many cross-vendor extensions coming to Vulkan, such as Vulkan Video, which will allow TouchDesigner to access the hardware video encoding/decoding of all GPUs with a single API in the future.
Some operations can be further threaded to improve performance, such as video input and outputs.
More operations within TouchDesigner can be done with compute shaders in the future. Features like NotchLC encode/decode on macOS and certain TOPs re-written using compute shaders, will all be possible.
Vulkan's better support of HDR color space outputs is another feature we can look forward to in the future.

Python Version

Python - The version of Python in TouchDesigner has been updated to 3.9.5. The previous version was Python 3.7.2. To discover 'What's New' in Python since then, refer to the Python 3.8 and Python 3.9 documentation.

High DPI Panel Rendering

A complete overhaul of panel component rendering introduces panels that are no longer rendered with fixed sized textures. Panel contents are drawn directly to screen now, enabling resolution independent rendering and perfectly crisp text in TouchDesigner UI and control panels at any zoom level.

TouchDesigner's UI now takes advantage of hiDPI and high resolution monitors. Text, borders, and lines are now drawn directly to screen taking advantage of all the pixels available, without the internal scaling that previously made the UI soft and fuzzy on hiDPI monitors.
Text COMP - A new component for rendering resolution independent text in panels, it uses the Slug Library and does everything on the GPU. Read more in 'New Operators' section below.
Built-in UI text rendering was also changed to use Slug Library and panel direct rendering, you will see this in parameter dialogs, nodes viewers and names, and built-in menus and pop-ups, etc.
DATs viewers have been updated to use this new rendering engine as well.
The rest of the user interface has been updated to use Text COMPs internally, bringing this great looking to dialogs, the palette, main menus, and timeline.
Widgets - Upgraded all fields and labels to use new Text COMP.

Steinberg VST® (Virtual Studio Technology)

VST is a proprietary standard for audio plug-ins that has made it possible for hundreds of third-party software programmers to create and sell virtual instruments files in popular digital audio workstations (DAWs), and now, TouchDesigner!

TouchDesigner now supports VST3 plugins via the Audio VST CHOP. Our implementation uses the JUCE framework for VST3 plugins.

Support for VST3 plugins

Any parameter of the VST plugin can be exposed as a TouchDesigner parameter.
The VST plugin's UI can be displayed in a floating window.
Filters (take input channels), Instruments (no inputs) and MIDI plugin types are supported.
Send MIDI Events to the VST plugin with the MIDI Event functions
Capture the MIDI events the VST plugin generates using Audio VST CHOP's callbacks DAT.

SDI and Video Pipeline Upgrades

Deltacast support has been added, TouchDesigner supports both Deltacast PCI-e video cards and the Deltacast FLEX product line.
12-bit workflows are now supported
AJA devices can now capture at 12-bit RGB 4:4:4 and output 12-bit formats when the device supports it.
Blackmagic Design devices can now output 12-bit formats from the Video Device Out TOP.
10-bit workflows have been improved
Video Device Out TOP - AJA 10-bit capture and output now works when using firmware that does not support color space conversions.
Video Device Out TOP - Fixed Blackmagic Design 10-bit RGB 4:4:4 output not working correctly.
Better performance when doing 10-bit input and output.
Improvements to Video In and Out
Video Device In TOP - Added 'inputSignalFormat' member, which can be used to drive the 'Signal Format' menu on a Video Device Out TOP.
Video Device In TOP - New transfer modes for Vulkan implementation.
Video Device In TOP - New 'signal_fps' Info CHOP channel for Blackmagic Design and AJA devices.
Video Device Out TOP / Video Device In TOP - Added 'Reset Stats' pulse parameters.
OpenColorIO TOP - Upgrade to SDK 2.1.0 which provides a better renderer able to read the latest .ocio config standard format. This offers new features such as built-in LUT referencing (for the most common LUTs) which can avoid having to reference lots of external LUTs.
NDI - Updated to NDI 5.0

Custom Operators

Custom Operators now support calling Python callbacks from the C++ code.
Custom Operators can now use the CPython API to define a custom Python class for the node. This allows directly interacting with the C++ code via Python calls.
The provided headers for Custom Operators are now put into the 'TD' namespace. Existing plugins will continue to work, but if you upgrade to the latest header files you'll likely need to add using namespace TD; in your code to find all the definitions when recompiling your plugin.
Custom Operator TOPs (CPlusPlus TOP)

Due to large changes in the switch to Vulkan, all Custom Operator TOPs (but not other node families) will need to be upgraded to a newer API. The API is much cleaner to work with and more powerful. Upgrading should be quick for cases other than TOPs written using OpenGL mode, which have no upgrade path yet, until we add a Vulkan mode. The CPU and CUDA modes are much cleaner to work in now though. New features that are now supported include:

Support for downloading and creating many more pixel formats.
Can now create 3D, 2D Array and Cube textures in CPUMem mode.
Support for uploading to multiple color buffers, with different dimensions/resolutions/formats when in CPUMem mode.

ParGroups

(Work In Progress)

A ParGroup is a group of related parameters offering a more natural way to work with the ParGroup as a whole instead of its individual parameters.

Often all parameters sharing the same line in a parameter dialog belong to the same ParGroup. For example, in the Transform TOP, Translate is a type of ParGroup, with two primary Par objects: tx, ty and a unit parameter. ParGroups with only single elements are also possible. For example, the Uniform Scale parameter of the Geometry COMP belongs to a ParGroup with only one parameter.

ParGroup - A new python object that allows you to work with parameters at a higher level.
ParGroup Execute DAT - A new DAT similar to Parameter Execute DAT but organized around ParGroup objects.
ParGroup_Class - A python class for accessing ParGroup objects.

New Operators

Text COMP - The Text COMP is used to display text in panels for building user interfaces and control panels. It is a replacement for the Field COMP, completely replacing it and adding many new features not previously available.
Works for both static and editable text.
Modes for displaying strings and multiline text, float and integer numbers, password fields and more.
Text formatting options include support for Python syntax, Custom C++ Fmt and Custom Python F-String syntax.
Number formatting options for displaying timecode, scientific, percent, and currency values.
Supports inline formatting directives in the text for things like color, strikeouts, underlines, smallcaps, subscript, gradients, etc. For example, the code {#color(255, 0, 0, 255);} will turn all text that follows on that line red.
Resolution independent text drawn directly to screen at the resolution required for the panel size, resulting in perfectly crisp text at all zoom levels.
GPU based using the Slug Library.
Geo Text COMP - A new component that creates text geometry directly on the GPU using the Slug Library, similar to the 2D Text COMP.
Stype Out CHOP- A new CHOP that outputs Stype formatted camera tracking data.
FreeD Out CHOP - A new CHOP that outputs FreeD formatted camera tracking data.
ParGroup Execute DAT - A new DAT similar to Parameter Execute DAT but organized around ParGroup objects.

SDKs Updated

CUDA 11.2
NDI 5.0
OpenCV 4.5.2
Slug Library 6.0
OpenColorIO 2.1.0
Substance SDK 8.3.0
Blackmagic Design SDK 12.2.
Material Design Icons SDK 5.4.55. Now includes cheat-sheet in installer.

New Features

Network Comments, Boxes, and Annotations

(Work In Progress)

Add Comments and Boxes to your networks to keep them organized and easy to understand.
From the Network Editor RMB menu, select 'Add Comment' or 'Add Network Box' to add one to your network.
Change color using the Color Swatch in the Network Editor (press c-key to display).
Right-click to open the parameters.
Comments and Network Boxes are referred to as 'Utility Nodes' and do not show up in OP Find, COMP Class.findChildren(), network list mode and other searches by default.
Annotate COMP - Fully customizable network tools can be created with the Annotate COMP.
Embed operator viewers and panels in the network editor background, control options for interaction and enclosing operators, and specify depth placement amongst network elements.
Comes pre-configured but can also be built from scratch as a custom tool by resetting parameters and deleting its contents.
Pre-configured Annotate COMP includes more settings for text, color, size, and OP Viewer in the background.
Ouster Support

Ouster now requires firmware v2.x and with this TouchDesigner now supports all Ouster models.
Added option to not display incomplete frames.
Added support for signal multiplier. Requires firmware version 2.1 or greater.
Output now reverts to a default black texture when the active flag is disabled.
Interpolation and Mipmapping Controls

Panel COMPs - All Panel COMPs now have 'TOP Smoothness' parameter which controls background TOP's viewer smoothness settings. In previous builds of TouchDesigner, this was always 'Mipmap Pixels', so old files will load with that setting whereas the default for new Panel COMP's is 'Interpolate Pixels'.
Null TOP / Out TOP - And other all other TOPs can now select their 'Viewer Smoothness' and 'Fill Mode'. Previously this was not available on 'instance-style' TOPs ie. Nulls and Outs.
TOP preference - Default values for 'Viewer Smoothness' and 'Fill Mode' can now be controlled by a preference.
Pre-Multiply by Alpha Overhaul

The way TOPs and MATs deal with Pre-Multiplied Alpha has been overhauled. Now they will use Pre-Multiplied Alpha when generating colors, and the MATs assume incoming textures and colors are Pre-Multiplied Alpha.
Camera COMP / Light COMP - You can now control Pre-Multiply by Alpha settings for background color. On by default for new nodes.
Fit TOP, Corner Pin TOP, Layout TOP, Level TOP and Transform TOP - Added 'Pre-Multiply' controls for these operators.
Miscellaneous

FBX COMP - Added 'Texture Directory' parameter which is used as an additional search location for external texture files if they can't be found at the default location specified inside the FBX file.
Timer CHOP - Added a new menu 'Length Type' which specifies Fixed or Infinite length. When set to 'Infinite' length, time counts without limit. This is to enable Timer CHOP to drive for generative components that have no fixed length.
Added information about SDK version used by the operator. MMB on an operator or click "i" Info icon in the parameter dialog to view in the pop-up info.
New Search Dialog that also offers Search & Replace functionality. Available globally or per component. Massive time-saver and helps you gain insight in your large projects.
New Operator Snippets

Lots of new examples highlighting the new features in Experimental. Open Help > Operator Snippets and press the 'Newest' button. (Note: Web Client DAT example 5 hangs)

New Python

Camera.frameBounds() - Now returns a camera frame width that can be used for the Ortho Width parameter of an orthographic camera.
COMP Class.reload(path, password=None) - Reloads the component from the given file path. This will replace its children as well as top level parameters and update flags, node width/height, storage, comments and inputs (but keep original node x,y).
Connector Class.description - A description for this connection. Example: 'Color Image'
Dependency Class callbacks method
A modifiable list of functions. When the Dependency object is modified, it calls each function on the list.
The function is called with a single argument which is a dictionary containing the following:
'dependency'- The Dependency that was modified.
'prevVal' - The previous value if available.
'callback' - This callback function, which can be removed the the Dependency callbacks if needed.
NetworkEditor.placeOPs method accepts opType for creating node. This argument can be either a string or a type. Example: ui.panes[0].placeOPs(opType='waveCHOP') or ui.panes[0].placeOPs(opType=waveCHOP)
ParGroup_Class - A python class for accessing the new ParGroup objects.
tdu Class - Added tdu.forceCrash() to force a crash for debugging and crash recovery testing.
tdu.tryExcept(func1, func2 or value) -> result - A 1-line try-except syntax. Second parameter can be another callable function or a direct value.
Example:

tdu.tryExcept(lambda: 1/me.par.w, 0.0) # second argument is simply 0.0 tdu.tryExcept(lambda: 1/me.par.w, me.GetDefaultValue) # Good: me.GetDefaultValue not called until needed. tdu.tryExcept(lambda: 1/me.par.w, me.GetDefaultValue()) # >> INCORRECT <<. Always calls second function even if not needed.

UI Class.rolloverPanel returns the latest panel to get a rollover event. Takes into account click through, depth order, and other panel settings.
WebclientDAT Class.connections - List of current HTTP connection identifiers.
WebclientDAT Class.closeConnection(id) - Close the connection associated with the given identifier.
WebclientDAT Class.request - Added timeout keyword.
Added support to assign a list of OPs to a multi-OP parameter in python.
calling eval() on multi-op parameters will now return a single op if there is only 1 result, a list if there are more than 1, and none if there are no results.

New Palette

TDAbleton - Version 1.33. Components are now available individually in the palette. TDAbleton now supports both Live 11 and Live 10.
Parameter option to suppress disconnect error flag.
Utility parameter options to load Live sessions directly from TDA.
Better handling of duplicate parameter names.
BACKWARD COMPATIBILITY - cuePointNum in song callbacks now holds index into cuepoint table.
Palette:materialDesignIcons - New component replacing symbolPicker.
palette:vstHost - New component that loads VST3 plugins into the Audio VST CHOP and adds examples for MIDI keyboard in, a MIDI sequencer, and VST automation using the Bind CHOP.
Palette:kantanUVHelper - Fixed an issue where kantanUVHelper could fill GPU Memory considerably.
Palette:presets / Palette:particlesGpu - Fixed an issue where custom user presets wouldn't be recalled properly.
General updates and tweaks
palette:lister 1.11
palette:treeLister 1.6
palette:searchReplace
palette:operatorPath - The operatorPath Component is a full-featured UI element for navigating operator paths in TouchDesigner networks. It is similar to the path bar at the top of editor panes.

Bug Fixes and Improvements

Animation COMP - Preserve custom columns in keyframe DAT tables when editing keyframes.
Button COMP / Slider COMP - Updated to use Text COMP internally and include 'Label' and 'Value' parameters.
List COMP - Improvements
Fractional font sizes are now allowed. Font sizes are no longer rounded down to the nearest integer. See Backward Compatibility below.
Text offset now supports fractional floating point numbers.
No longer wraps around negative indices when accessing cell attributes.
str(ListCell) returns calculated text at that cell, or blank.
Fixed dragging onto list components when their viewer has been resized/stretched.
Fixed dropping to use python defined items.
Callback DAT is now docked on the outside of the OP by default.
Added support for custom fonts using Font File attribute.
OP Viewer COMP - New 'Interactive' parameter controls interaction with mouse and keyboard.
Panel COMP - The Drag-and-Drop callback onStartHoverGetAccept now contains information as to where the mouse entered the panel in its info dictionary. Key name coords.
Panel COMP - 'Allow Expansion' parameter controls whether or not parameters can be expanded to reveal expression fields.
GLSL MAT - Point Sprites must now use TDPointCoord() instead of gl_PointCoord to obtain the texture coordinates for the sprite. See Backwards Compatibility section below.
Level TOP - Defaults changed so floating point input values are no longer clamped.
Luma Level TOP - Fixed alpha being effected by options other than 'Opacity'.
Ouster TOP - Fixed missing and skipped packet numbers incrementing incorrectly when using a partial azimuth window.
Switch TOP - When in Blend mode, will no longer cook 2 inputs if the switch value is an integer.
Text TOP
Added smart quotes support.
Added support for floating point positions offsets for non-bitmap fonts.
Stroke rendering now looks a bit better, but still not as good as Scalable mode.
Video Device In TOP - Reduce stalls that occur when the input signal format into Blackmagic devices changes.
ZED TOP - Now outputs a blank image when there is an error or no camera found.
Joystick CHOP - Added 'Active' toggle.
MIDI In CHOP - Much smoother timesliced channels especially when dealing with high frequency input.
MIDI Out CHOP - Fixed issue with random values being generated when number of channels changed.
Panel CHOP - Fixed crash when target panel deleted.
Select CHOP - New options to filter channel names by numeric suffix.
Sync In CHOP - Warning when project still in Realtime mode.
Sync Out CHOP - Include client Realtime status in info.
Timer CHOP - Fixed calling initialization when called from onSegmentExit callback.
Art-Net DAT - Fixed undercooking while polling and not visible in network.
Examine DAT - Fixed missing entries with matching values but different keys.
OP Execute DAT - Fixed onDelete() never being called.
Parameter Execute DAT - Fixed failing to monitor changes when a parameter default is changed.
Using floating point values in integer parameters will now round the value (either through expressions or exports), instead of truncate it. See Backwards Compatibility section below.
Added information about SDK version used by the operator. MMB on an operator or click "i" Info icon in the parameter dialog to view in the pop-up info.
Kinect Azure TOP / Kinect Azure Select TOP / Kinect Azure CHOP - Added SDK version to the pop-up info.
Nvidia Denoise TOP / Nvidia Background TOP / Face Track CHOP - Added SDK version to pop-up info.
Automatically created parameters will no longer be classified as 'custom parameters'. This applies to Parameter DAT filters, dialog options, and python access. Affects following operators: C++/Custom Operators / Audio VST CHOP / Substance TOP / Notch TOP
Added drag-drop support for json file extensions
Fixed color picker dialog parameter UI when dealing with ranges other than 0-1.
Added wildcard dependency to ops function search so new nodes will trigger a recook.
Fixed issue with self parameter dependency (me.X depending on me.Y) not getting updated when parameter Y is updated during cook.
Allow binding to named object attributes. Expression format is 2 tuple: (object, attribute), example: (op('box1'), 'nodeY')
Allow binding to python None value, which return blank string or 0. Allows binding expressions to evaluate to None, without error when source not available.
Fixed Bind expressions involving 'me.curPar'
Fixed some issues with menu dropdown lists not updating correctly in custom parameters.
ctrl+c now interrupts Python scripts in macOS. Try it for hangs.
Refactoring panel layout for more efficiency and better performance.
Reconnecting audio devices better handled by Audio CHOPs.
Improvements to pop-up help performance and size for DATs.
toecollapse/toeexpand now support duplicate operator names in which only case changes.
Optimizations in cooking, data dependency improvements, updating parameter value optimizations, extension initialization improvements.

Backward Compatibility

BACKWARD COMPATIBILITY - wrnchAI CHOP - Has now been disabled due to the company no longer selling licenses.
BACKWARD COMPATIBILITY - Using floating point values in integer parameters (either through expressions or exports) will now round the value, instead of truncate it. This may cause different behavior for fractional values exported to menus or other integer parameters.
BACKWARD COMPATIBILITY List COMP - Fractional font sizes are now allowed. Font sizes are no longer rounded down to the nearest integer. Example: Exporting 4.9 to an integer parameter will now be considered a value of 5, not 4.
BACKWARD COMPATIBILITY - GLSL MAT - When working with Point Sprites, you must now use TDPointCoord() instead of gl_PointCoord to obtain the texture coordinates for the sprite.
BACKWARD COMPATIBILITY - PBR MAT - Removed 'Mult Alpha by Light Luminance' feature, as it didn't work with the more alpha-correct-behavior of the shader now.
BACKWARD COMPATIBILITY - The python object type for the Nvidia Flow Emitter COMP has been renamed to flowemitterCOMP from flowEmitterCOMP to be consistent with naming conventions.
BACKWARD COMPATIBILITY - Light COMP - Removed defunct 'Shadow Mapping Projection' view type from projection menu.
BACKWARD COMPATIBILITY - SVG TOP - Has been removed for now because the OpenGL feature it was based on does not exist in Vulkan. Please use Palette:webSvg to load SVG files in TouchDesigner.
BACKWARD COMPATIBILITY - The tscript command 'jpeg' has been removed. The 'quality' keyword in OP.save() can be used to control JPEG quality instead.
BACKWARD COMPATIBILITY - Script OPs that were accessing the script methods outside the onCook callback must now be locked.
For example, calling things like scriptCHOP.appendChan() from outside the scriptCHOP's onCook() method is no longer allowed, since it caused random behavior. Now if those methods must be called, lock the Script CHOP first. This goes for any of script OP methods that modify operator content.