DAWbench - Reference Benchmarks :.


Cross Platform DAW Performance - Part VI:


For those who have been following the series of reports up until now you will be aware that the focus has been on cross platform DAW hosts utilizing a selection of DSP plugins. The reports have definitely brought up some points of interest and discussion , but as I am reminded quite regularly , DSP plugins are only 1/2 of the equation , Virtual Instrument / Sampler performance is becoming far more pertinent to many users.

I have had a new Virtual Instrument / Sampler benchmark in development for quite a while but had never seemed to find the time or energy to finalize it. Some of the initial hurdles were based around trying to offer a test session using freeware players/libraries which was proving extremely difficult , and a moving target at the best of times in terms of availability/stability , so I just kept putting it off.

With the release of PT9 and the incredible momentum it generated in the Audio community , the timing was ripe to get the new test session finally sorted so that we could do some direct head to head with AVID's proprietary RTAS format, and VST/AU in regards to Virtual Instrument scaling/performance. As I noted in the conclusion of the last report , RTAS performed extremely well in the DSP based tests , but VI's could be a totally different story. There has been many reports of poor comparative VI performance within PT over the years, but never anything that could be presented in a quantifiable and empirical manner , so it would definitely be interesting how it would play out when we placed it under the microscope.



Preparing for Battle :.    

First call was to choose the Virtual Instrument / Sampler that would be the driving engine of the session , and for this I chose Kontakt 4 , which is arguably the most used and stable of all sample base Virtual Instruments on the market.

Also using the included library gave me the required scope of instrument sounds for the musical content portion of the test session, and its something that a lot of end users would have access to.


Test system was again the tried and true i7 Development system as used in all previous testing , and the RME HDSPe AIO card was chosen for its proven reliability and performance .

Operating systems and DAW versions were maintained from the last test runs to ensure absolute consistency with the previous reports , but future testing will be moved to later versions of the DAW Hosts and operating systems.


DAW Benchmarks:
DAWbench VI Universal - 2011 .

DAW Application Details :
AVID Protools : Version 9.0.x
Steinberg Cubase 5.5.x :

Reference System Detail:
Intel i7 920 Quadcore/ 2.66 GHZ/
Intel X58 / 6 GB DDR3-PC10600.

Audio Hardware Detail:
RME : HDSPe : Driver 3.0x

O.S Detail:
Windows 7 x64 / OSX 10.6.2

DAWbench VI - Universal : Detail and Methodology :

On this initial testing run only Cubase and Protools were used , being the 2 most popular cross platform DAW hosts I felt they were the best starting point. Other DAW hosts will be included as we progress with future testing.

Test session detail:

There are 17 Stereo Audio Tracks - 16 Sine Waves, 1 Drum Loop.

16 Midi Tracks in the musical template with melodic and sustained parts , all playing individual parts of a Kontakt 4 Multi Setup.

There are 6 instances of Kontakt 4 all running 16 individual parts, the first instance has the orchestral and band instruments for the musical content of the session, the remaining 5 instances all have 16 independent parts that are assigned to the polyphony load tracks, each part having 20 notes of polyphony in sustained sections that correlate to the musical context.


The polyphony parts are all zeroed out volume wise tho, as it would turn into a mess pretty quickly.

There are 2 versions of the test session , one with convolution verbs on the orchestral elements - CV, and one without - NCV.

The session as it stands allows an extra 1600 voices of polyphony above the template, which on testing will have enough headroom for even the fastest of the current dual hexacores.

Each instance of Kontakt is assigned 320 notes of polyphony. Memory used will be around 2.7GB, so this test is not suitable for 32 bit systems.

Methodology for running the test is very straight forward.

First off the latency buffer of the audio interface is set to its lowest setting and then the Host DAW is launched and session loaded


While the session is running, I un mute each polyphony track in turn, each time adding 20 notes of sustained polyphony.

On hearing any crackles or drop outs, I would then move back to the previous track , mute and un mute that track again just to ensure that the previous setting had played back cleanly.

I would then save each session , exit the application, readjust the latency setting of the audio interface one step higher , reload session and continue incrementally increasing the polyphony.

I repeat the above at each latency setting until I have completed testing the session at all required latencies.




DAWbench VI - CV/NCV : Kontakt 4 : RME HDSPe AIO :

Looking at the results it is clear that this a completely different playing field in comparison to the DAWbench DSP test results.

There are 2 aspects evident , one being the huge discrepancy between Cubase performance on Windows and OSX, even more so than on the DSP based test and also Protools/RTAS failing dramatically across both platforms. Its obvious that Virtual Instruments/Samplers are a completely new ball game when it comes to RTAS, and all of the past grumbling of poor performance bubbles back to the surface.


A quick round up of the comparative results for Cubase ranging from 064 thru to 512 Samples is 140% - 216% in favor of Windows, which at the higher latencies equates to a huge variable in the available polyphonic capabilities .

The respective Protools results range from 60-400% on the latencies that were available in favor of Windows, but to be realistic , the results were so abysmal it doesn't really give us a lot to go on in respect to cross platform performance when the figures are placed in perspective and also in comparison to Cubase.


The comparative results for Windows range from 250-300% in favor of Cubase , on OSX the variable ranges from 40-200% again in favor of Cubase.

We need to note that the performance of Cubase on OSX is anything but good, so the results for Protools are even further compromised.

Conclusion :

I started writing this report in January 2011 not long after the release of Protools 9, but with my attention focused on other areas , 10 months passed and I had yet to get the report completed and uploaded.

With the release of new versions of both Cubase ( 6) and Protools (10) in 2011 the report is a little dated, but can be used as a comparative against the newer versions to see if anything has shifted in regards to overall performance.

With that out of the way , what conclusions can be drawn past the fact that RTAS performance in regards to Virtual Instruments is a significant problem that AVID seriously need to address.

Seeing the results in graphical form is indeed imposing for those that have only had a passing interest in the past and probably even more so for those that are weighing up the option of Protools for a larger compositional working environment where VI performance is paramount


Of course the argument can be made that the results just using Kontakt 4 are in no way conclusive , and some have even suggested I should be using AVID's own Structure sample based Virtual Instrument to get a fairer comparative of the potential of RTAS VI performance

The obvious problem with Structure is that it is Protools only and secondly does not have a wide user base or available libraries, so comparatives in other DAW hosts is impossible. Also I needed an Instrument that is cross platform , multi format, is widely used, and is known to be stable and efficient - Kontakt fills all of the above criteria.

I have already had numerous punters on the open forums point out this isn't news to anyone that has used Protools in that capacity. Some believe that I am simply beating a dead horse , and that there is a viable work around available using VEP Pro as a hosting environment using VST /AU version of the virtual instruments , so why am I bothering ?


Simple - as good as the available workaround of using VE Pro is , it doesn't negate the fact that native Virtual Instrument performance in Protools needs to be addressed.

Judging by reports I have read recently from AVID representative, its isn't going to be anytime soon , especially with the shift away from RTAS to the new AAX plugin format introduced in Protools 10.

This is only a preliminary report, the test pool will be expanded to also include Reaper and Studio One V2 as well as the new AAX plugin format.

I am yet to see any announcements from VI developers in regards to AAX , but the DSP plugin developers are falling in line pretty quickly , so I am hoping to have some comparatives of the new AAX format in the mix with the established reference plugins in the following report.

Vin Curigliano
AAVIM Technology
November 06 2011

Part I | II | III | IV | V

© AAVIMT 2011