imagomat
     
 
The Making of VizKit

I ended support for iTunes visualizer development in 2017.
This page is for historical reference only.

Set Up

VizKit is written in C/C++. It is built as a shared library. The product is a dll file on Windows and a Mach-O shared library bundle on Mac OS X.

VizKit receives and processes messages by iTunes as outlined with Apple’s "iTunes Visual Plug-ins SDK" (iTunesVisualPlugInsSDK2_0_1.zip - Jan 30, 2015) which can be downloaded at https://developer.apple.com/downloads. The SDK "contains the files necessary for iTunes Visual Plug-in development for iTunes 10.4 or later, and includes documentation and sample code. Specifically, this SDK contains the files 'iTunesAPI.h', 'iTunesVisualAPI.h', and 'iTunesAPI.cpp' that developers will use to develop their own plug-ins. The sample code is a fully functional Visual Plug-in developed for Mac OS X 10.5.8 and later with Xcode 3.2 and for Windows XP SP3 and later with Microsoft Visual Studio 2013. Both 32-bit and 64-bit versions of iTunes are supported." Apple’s SDK also contains TN2016.pdf, a tech note dated 2011-07-19, which describes iTunes Visual Plug-ins development.

Apple supports third-party iTunes Visualizer development since iTunes 1.1 (June 2001). VizKit was first released on June 8, 2004.

VizKit is a complete samplework for iTunes Visual Plug-in development. In addition to Apple’s SDK it contains OpenGL graphics, configuration dialog integration, convenience functionality, and extended documentation.

Main Idea

VizKit consists of various functions and classes assembled together. The main functionality is centered around the VisualEnsemble and the VisualActor class. The VisualEnsemble is the group of all actors. It keeps track of the actors and calls each actor during the show. The VisualActor class is a lean abstract class that defines the interface for each concrete actor. Each specific actor inherits a method getName(), a method getState(), and a method show(). Show() must be overridden by each concrete actor, while getName() and getState() do not need to be implemented by the concrete subclass. By default getName() returns the value of the protected instance variable actorName – each specific VisualActor is responsible for setting its own inherited actorName variable to the appropriate value. By default getState() returns the value of the protected instance variable state; each specific VisualActor is responsible for setting its own inherited state variable to the appropriate value. The VisualEnsemble calls the show() method of each actor while displaying the visualizer. Show() is the main render and display routine of the VisualActor that gets called repeatedly with a fast refresh rate.

Placed above the VisualEnsemble and the VisualActors, the VisualStageControl coordinates the show on the screen. The VisualStageControl can manipulate the behaviour of individual VisualActors by evaluating each visual actor’s name.

diagram

Customization

To use the VizKit samplework, you have to subclass and implement the VisualActor interface. A sample VisualActor implementation is already included with the VizKit source code files: the TemplateActor.

The action performed by the TemplateActor is pure sample work. It can be changed at will.

The TemplateActor is a lightweight implementation of the VisualActor interface. It includes source code that draws dynamically created bitmap image data as part of an asset. It also fetches network data, names of musicians whose work is similar to the music which is currently playing (by means of the last.fm web service at ws.audioscrobbler.com). The artist names are used to create a bitmap image which is displayed at the right side of the screen.

To customize the VizKit samplework with the goal of ending up with an independent and individual iTunes visualizer, settings in VisualCustomization header and implementation file need to be customized as documented inline.

If you want to display a configuration dialog, the dialog with its resources and actions needs to be customized as well.

Persistent Preferences

VizKit provides means for storing and retrieving preference values. Identifiers for preferences are declared and defined as enum values in the public section of VisualPreferences.h. Preferences can be of data type int (PreferenceKeyInt), of data type float (PreferenceKeyFloat), of data type char (PreferenceKeyChar), or of data type bool (PreferenceKeyBool). The values of the identifiers are initially set to their factory defaults with the function setDefaultValues(). A current preference value can be retrieved by calling getValue(). On runtime a preference value can be set by calling setValue(). The preference values are persistently stored after calling storeValues() method.

Each preference key added must be appended to the list of keys in the appropriate conversion function where the key is translated to the name and vice versa (VisualPreferences::convertPreferenceKey[Int|Float|Char|Bool]StringToKey() and VisualPreferences::convertPreferenceKeyToString()).

Persistent preferences retain their value across iTunes sessions (after quit and relaunch).

Mac OS X: The preferences are stored in the file "de.imagomat.vizkit.plist". The file is located inside the preferences folder of the user home directory ("~/Library/Preferences/").

Windows: The preferences are stored in the file "Preferences.txt". The file is located inside the "\Documents and Settings\{username}\Application Data\Imagomat\VizKit\" directory of the system volume. To see and access the file, “Show hidden files and folders” might need to be turned on (View settings of “Windows Explorer”’s Folder Options).

In case of problems with preferences, the preference file can always be deleted again. The visualizer will return to use its default preferences if it does not find a preference file.

By editing VisualNotificationKey.h, own notifications can be added to a custom visualizer. Custom notifications must be added to the switch block in VisualMainAction::renderAction().

The Notification is the Message

VizKit uses messaging to notify Actors about changes of data or events. Messages and events are passed to VisualActors by invoking VisualActor's interface method handleNotification() with a VisualNotification as parameter. Message keys are defined as VisualNotificationKey. Each VisualActor that wants to be notified about events has to register for them. The registration can be easily done in VisualActor's virtual init() method by calling VisualNotification::registerNotification() with the instance of the VisualActor and the VisualNotificationKey as argument.

VisualNotifications convey messages. Messages can be event information. They can also include additional message data. At minimum a VisualNotification contains a VisualNotificationKey. To send a notification, the post() method has to be invoked. VisualNotification’s static method post() allows the distribution of a VisualNotificationKey. To send a VisualNotification with additional data, the public method post() has to be invoked on the instance.

In a typical scenario a VisualNotification acts as a transmitter of a changed preference value.

All newly introduced notifications must be added to the main switch block in VisualMainAction::renderAction() and to the conversion function VisualNotification::convertNotificationKeyToString().

The Canvas is the Frame of the Action

When setting up the scene, iTunes passes a platform-specific view to the visualizer. On Mac it passes an NSView*, on Windows a HWND gets passed. On the Mac, a subview is added to the view. The refresh rate on the Mac is bound to a CVDisplayLink. On Windows the screen refresh is triggered by iTunes via the visualizer interface calls.

As a consequence of this setup, on the Mac the VisualMainAction::renderAction() call is performed on a concurrent render thread, while on Windows the rendering is executed on iTunes’s main thread.

Canvas dimensions are measured in pixels.

User Space is (Coord-)Place

The viewport of the OpenGL surface is set up with the two dimensions of the canvas (glViewport with width and height). Since OpenGL is designed for 3D graphics, the surface is further transformed to a frustum that can have an orthographic or perspective projection (glOrtho, gluPerspective, glFrustum).

The projection transformation is modelled with the VisualCamera. The user space is measured in coordinates. Coordinates are fictional units. They are not directly mapped to pixel values or distances in inches.

The center of the user space is 0.0, 0.0. If the canvas is a square the coordinates are -1.0 to 1.0 from left to right, and -1.0 to 1.0 from bottom to top. More often than not the canvas is a rectangle, so that the width and height values of the viewport are compared and the smallest border is laid out from -1.0 to 1.0. The wider border gets scaled according to the aspect ratio. Near and far values on the z-axis are set to 0.0 to 10.0 with orthographic projection. With the perspective projection the position of the near clipping plane can be set; the far clipping plane is positioned as far as the longest vertical or horizontal distance.

Basics

VizKit provides basic functionality for string, file and image operations.

Animated Properties

One of the main tasks of an audio visualizer is animation. VizKit aids in animating properties of graphical objects with the VisualAnimation and the VisualAsset class. An animation has a start value, an end value, and a duration. One animation can only animate one property (AnimatedProperty). So the type of animation (the property which is supposed to be animated, e.g. size, opacity, etc.) is set up when initializing an animation. To reduce complexity the animation is tied to an asset. Depending on the type of animation, the running animation calls a callback function of the visual asset. Within VisualAsset’s callback function the property of the asset is calculated according to the current value of the animation. When executing VisualAsset’s draw() method the animated value of the property is taken into account.

Tools and Utilities

The VizKit samplework is equipped with a number of tools. The VisualTiming class assists by taking time values. VisualErrorHandling’s writeLog() function prints strings to a log file for debugging and diagnostical purposes. When not printing into a log file on the desktop, on Mac OS X messages are printed to console.log while on Windows the messages are added to the event log.

The ProcessMonitor class is no tool but a VisualActor on its own. It prints strings on the canvas. The process monitor strings can be hidden or shown by pressing the <s> key on the keyboard. By calling the function setProcessInfo(), a label and a string can be passed to the ProcessMonitor. The function call is dispatched to the VisualStageControl which passes a notification to the VisualEnsemble. The notification is received by the ProcessMonitor and with each show() call the current values are presented on the screen. In addition to string values the VisualProcessMonitor shows a visual waveform representation of the current audio data and a spectral audio analyzer bar graph. The ProcessMonitor also shows a progress meter that displays the current play position of the currently playing audio track.

Track Info Display

The main task of the visualizer is the combination of the audio and the graphics domain. Audio data is processed and analyzed by the VisualAudioLab. Graphic operations are bundled with the VisualGraphics class.

Track title, lyrics and the cover artwork are essential metadata of an audio track. VizKit supplies functionality to display this data.

If the currently playing audio track contains cover artwork data, it is displayed on the canvas. The CoverArtActor imports the image data and applies custom transformations. The action of the CoverArtActor is only an example of what can be done with the image data of the cover artwork. The performance of the actor can be modified at will.

The Unicode-savvy title of the currently playing audio track is transformed into a texture memory image by the TrackTitleActor. The font rendering is done by loading a custom font which is bundled as a resource of the plug-in. VizKit uses the free OpenType font “Devroye” by Apostrophe; but any other free font can be used as a plug-in resource. System-wise available fonts can be used of course, too. And the content of the textured string also can be set at will.

A texture of the lyrics text is created by the TrackLyricsActor. Like TrackTitleActor it also uses VisualImage’s initWithStyledString() method to create the texture of the string.

Options dialog

VizKit implements a cross-platform options dialog with a tab layout. The first tab pane displays about information for the VizKit visualizer. The second pane allows to set the font face for the track title display. And the third pane sports a checkbox for activating and deactivating the automatic check for an available update online.

The cross-platform function VisualConfigurationDialog::show() hides the platform-specific implementation of the options dialog.

Building

The distributed files do not have prebuilt binaries with the Xcode project or the Visual Studio project.
The visualizer product has to be built with the provided resources and source files by means of the provided project files with the platform specific IDE.

Project Options

On Windows the sources for 64-bit Release are compiled with these project options:

/GS /W3 /Zc:wchar_t /I".\..\..\..\" /I".\..\..\source\" /I".\..\..\source\iTunesVisualAPI\" /I".\..\..\source\Actors\" /I".\..\..\source\Actions\" /I".\..\..\source\AudioLab\" /I".\..\..\source\Tools\" /I".\..\..\source\Tools\Timing\" /I".\..\..\source\win\" /Zi /Gm- /Fd"Release64\vc120.pdb" /fp:precise /D "TARGET_OS_WIN" /D "WIN32" /D "NDEBUG" /D "_WINDOWS" /D "_USRDLL" /D "WINVER=0x0501" /D "_WIN32_WINNT=0x0501" /D "_CRT_SECURE_NO_DEPRECATE=1" /D "_SECURE_SCL=0" /D "_AFXDLL=1" /D "_VC80_UPGRADE=0x0710" /D "_WINDLL" /D "_UNICODE" /D "UNICODE" /errorReport:prompt /WX- /Zc:forScope /GR /Gd /MD /Fa"Release64\" /EHsc /nologo /Fo"Release64\"

The library dependencies:

mfc120u.lib, mfcs120u.lib,gdiplus.lib, opengl32.lib, glu32.lib

On Mac OS X “TARGET_OS_MAC=1” is set as Preprocessor Macro.

On Mac OS X iTunes and the Finder need the PkgInfo file. Only when the PkgInfo file is accessible in the plug-in bundle directory (/Contents/PkgInfo), the bundle will be shown by the Finder as an iTunes plug-in file. The PkgInfo file identifies CFBundlePackageType (“hvpl”) and CFBundleSignature (“hook”) to the Finder so that the bundle can be presented as an iTunes plug-in file with the appropriate icon.
Since version 1.2 Xcode is able to generate PkgInfo file automatically. The build setting “Force Package Info Generation” has to be set (i.e. “GENERATE_PKGINFO_FILE=YES”). Prior to version 1.2 the PkgInfo file had to be copied manually or by script into the created bundle directory structure. If the bundle directory structure still does not present itself as a single file logging out and logging in again may force a refresh of the Finder view.

These source code files are set to File Type “Objective-C++ source” in Xcode on the Mac: VisualGraphicsCore.cpp, VisualConfigurationDialog.cpp and VisualView.m.

Visual Studio components

VizKit is built with Visual Studio 2013 with configuration Debug or Release for platform Win32 or x64. To use the built dll on Windows where no Visual Studio is installed, you might need to install additional components on the system. "Visual C++ Redistributable Packages for Visual Studio 2013" (vcredist_x86.exe) installs msvcr120.dll which enables Windows to run the Visualizer dll built with VS 2013.

Dependencies

Dependency on OS technologies is kept small. Only implementation files are supposed to include more OS dependent files like <windows.h> on Windows or <Carbon/Carbon.h> or <CoreFoundation/CFString.h> on Mac OS. OS dependent UI technology is only used for the Options dialog which is abstracted by the VisualConfigurationDialog class.

Notice

The iTunes visualizer API was broken with release 12.6.0 on March 21, 2017. Apple fixed this with release 12.6.1 on May 15, 2017.

Dependencies

Dependency on OS technologies is kept small. Only implementation files are supposed to include more OS dependent files like <windows.h> on Windows or <Carbon/Carbon.h> or <CoreFoundation/CFString.h> on Mac OS. OS dependent UI technology is only used for the Options dialog which is abstracted by the VisualConfigurationDialog class.

 

Documentation of classes and methods download chm file

 

changelog