Cross Platform DAW Performance - Part V :

AVID's release of Protools 9 has been one of the most anticipated and surprising moves in recent audio history , breaking away from the past model where the software was always locked to its own proprietary hardware and allowing the use of any 3rd party ASIO/Core Audio hardware has literally changed the DAW landscape overnight. Now I'm not saying they have done anything revolutionary, quite the contrary, for many they are still playing catch up in many areas , ADC/PDC on the Native platform being one example. However, unbolting the software and hardware has opened up opportunities for many end users where Protools was a secondary but necessary tool in the kit, to use their preferred audio hardware instead of what many considered an oversized copy protection device in the form of an interface that they only used to run the software.

That alone has caused a huge shift , not to mention the inclusion of ADC and most of the other toolkits as standard , shows that AVID have managed to turn the big ship around , are listening to their end user base, and are really trying to reinvent them selves after the previous M.O was proving to be on the nose for many. This is the first time I have been able to test Protools on a really level playing field in regards to preferred hardware , the M-Audio DAE testing showed some promise, so it was time to roll out ASIO/CoreAudio.

There really wasn't much at all to do to prepare for this round of testing as apart from Protools 9 , all of the results and testing has been carried over from Round 3.

In the last round I did shift to Cubase 5.5 and StudioOne 1.6 , but in regards to Cubase/Nuendo, the audio engine /optimization / performance is near enough to identical , they are basically the same application after all. In that respect, the earlier N5 results were fine to recycle

In regards to StudioOne, I went to 1.6 after an email from the dev's informing me of improvements over 1.5 in multiprocessor threading routines, but I didn't experience any improvement in the empirical environment of the test sessions , so I had no problem using the earlier results for 1.5 again for this report

All that was needed was to ensure that Protools was working as designed with the RME HDSPe under both ASIO and Core Audio.

After configuring the application and running some preliminary tests across both Win7 x64 and OSX to ensure no obvious surprises needed to be navigated, I simply knuckled down and for the first time was able to run Protools with the reference hardware and driver protocols used for all of the other DAW's in the test pool.

This is a truly level playing field , and something that for the longest time, many were never expecting to happen.

Lets see if AVID have managed to pull a rabbit out and go toe to toe with the other DAW's on their home ground.

DAW Benchmarks:
DAWbench DSP.

DAW Application Details :
AVID Protools : Version 9.0.x
Cockos Reaper : Version 3.6.x
Presonus StudioOne : Version 1.5.x
Steinberg Nuendo : Version 5.0.x

Reference System Detail:
Intel i7 920 Quadcore/ 2.66 GHZ/
Intel X58 / 6 GB DDR3-PC10600.

Audio Hardware Detail:
RME : HDSPe : Driver 3.0x

O.S Detail:
Windows 7 x64 / OSX 10.6.2

Lets take a look at the respective results.

Protools 9.0 : @032 no comparative was available as that setting was unavailable on OSX, @064 performed better on Windows by 29%, @ 128 performance was better on Windows by 6% @ 256 performed better on Windows by 2%

Nuendo 5.0 : @ 032 - performed better on Win7 by 201% , @ 064 - performed better on Win7 by 84% , @128 - performed better on Win7 by 48%,
@ 256 - performed better on Win7 by 31 %

StudioOne 1.5 : @ 032 - performed better on Win7 by 103% , @ 064 - performed better on Win7 by 104% , @128 - performed better on Win7 by 85%, @ 256 - performed better on Win7 by 81 %

Reaper 3.6 : @ 032 - performed better on Win7 by 40%, @ 064 - performed better on Win7 by 47%, @128 - performed better on Win7 by 47%,
@ 256 - performed better on Win7 by 49 %

On OSX, the results for Protool 9 were very similar to the performance of the DAE M-Audio results, being a lot more consistent across the board and easily outperforming all of the other applications @ the available buffer settings by quite a substantial margin.

The Windows results were the ones that were the mixed bag this time around. Protools 9 again showed all of the other app's a clean set of heels @ 032/064 and was looking like it was going to clean the slate if it continued to scale accordingly , but then @128 the scaling took a bizarre U turn.

My test procedure is to set the buffer to the lowest available, then incrementally load a session to break, save that session, exit out of the application, set the buffer to the next highest setting , re launch application reset audio engine at new buffer setting, load previous saved session and continue loading to break, save again, exit, reset, and so on from 032 thru to 256.

On this occasion tho, on opening the 064 session at the 128 buffer setting, I was confronted with a distinct red performance meter , and any attempt at playing the session instantly thru up a buffer warning. In all of the years I have been using this specific testing methodology, I have never experienced a saved session from a previous buffer setting not being able to be opened and run at the next highest buffer setting. To say it was out of the ordinary was an understatement.

My initial thought was that the audio engine had wigged out , so to double check that I exited out and reset the audio buffers to 032, and then opened the fully loaded session that I had saved @032 , fully expecting the audio engine to still be in a state of being inoperable . Of course that session loaded and played back fine, session is right on the limit, but playing back clean. O.K, exit out , change buffer to 064 , re launch PT9 , load the 032 session , no problem playing back. Close that session and open the even heavier loaded 064 session, plays back no problem , scratch and shake head, close session, shutdown Protools , shift buffer to 128, re launch PT9 , load session saved @ 032, no red performance meter, that a good sign I think to myself , press play, instant cough of the audio engine and buffer warning.. WTF ! ?

I had no choice but to start the 128 testing from a clean slate to see exactly where it was borking. As you can see from the above graph, the results @128 were down 10% from the 064 results , the 256 results did not even match the results I achieved at 032, let alone @064.

These results were defying any logic that I could attributed to how scaling has always increased respective of buffer size. I was starting to feel like I was in a Twilight Zone episode.. :-)

I had noticed on the MD5 results in the previous round that the foot had come off the pedal @128 in regards to scaling, but didn't really think too much about it, the scaling was at least still going in the right direction.

This session however was without saying, going against the norm of what I have experienced with this benching methodology in the 5 1/2 years I have been using it , and was not witnessed with PT9 using the DAE audio engine using the M-Audio interface.

I had witnessed odd behavior with plugin's on OSX during the testing , so I couldn't totally discount this being a strange buffering issue with the plugin itself, but because I hadn't experienced the issue with the M-Audio/DAE configuration , it was less likely.

Despite the U turn, comparative performance was still in favor of Windows across the board.

The next round with the mPressor would either confirm it as being an exception, or plunge us further down the rabbit hole.

We are now down to the last of the test session runs using the Elysia mPressor.

First off, the respective results across the 4 DAW applications

Protools 9.0 : @032 no comparative was available as that setting was unavailable on OSX, @064 performed better on Windows by 23%, @ 128 performance was better on Windows by 1 plugin ( .06%) @ 256 performed better on Windows by 2%

Nuendo 5.0 : @ 032 - performed better on Win7 by 237% , @ 064 - performed better on Win7 by 98% , @128 - performed better on Win7 by 71%,
@ 256 - performed better on Win7 by 71 %

StudioOne 1.5 : @ 032 - performed better on Win7 by 136% , @ 064 - performed better on Win7 by 96% , @128 - performed better on Win7 by 70%, @ 256 - performed better on Win7 by 97 %

Reaper 3.6 : @ 032 - performed better on Win7 by 61%, @ 064 - performed better on Win7 by 52%, @128 - performed better on Win7 by 48%,
@ 256 - performed better on Win7 by 49 %

On OSX, the results for Protool 9 were again very similar to the performance of the DAE M-Audio results, very consistent across the board and easily outperforming all of the other applications @ the available buffer settings.

The Windows results again showed that weird U turn @ 128 buffers , and again the results for 256 did not reach the results achieved @ 032 samples . At least we had some consistency, but that was not exactly a positive in this case.

The overall results on Windows again also showed a distinct advantage to the VST driven applications across the board, as witnessed with the results with DAE/M-Audio

The comparative performance again favored Windows , not by a huge amount but measurable up to 23% at the sweet spot of 064.

With 2 of the 3 sessions displaying an almost identical performance dip @ 128 , it would be harder to shift the blame to the respective plugin's. Of course it would have been more conclusive if all 3 displayed the same behavior , but there is still more likelihood that it is not plugin based , but more so an ASIO related issue, and could even be specific to RME HDSPe.

More investigation and testing will be required across a wider range of 3rd party hardware before the dust settles I suspect.

Well this round was definitely interesting, not only was it the first time that Protools was able to be tested in a direct comparative head to head with the preferred reference hardware , but also using the previously alien protocols of ASIO/ Core Audio.

Overall Protools 9 has been pretty impressive , not only has it maintained far better cross platform comparative performance using both DAE and ASIO/Core Audio, but it has performed admirably against the more established open hardware DAW applications and plugin formats. I think its safe to say that RTAS - for plugin's at least, is equally as efficient as VST in most cases on Windows, and more efficient on OSX. That is definitely a turn around to the pre conceived ideas I had coming into this testing.

Also the the other scenario of someone working @128, would have more headroom moving down in buffer size. It defies logic and it could point to some other under bonnet gremlins unless addressed .

I have forwarded the information to AVID development and will keep you informed if anything eventuates.

That rounds it off for 2010, I'll be back in 2011 with the some more fun and games.

Vin Curigliano
AAVIM Technology
November 15 2010

Part I | II | III | IV | VI