Cross Platform DAW Performance - Part IV :

In the last reports we covered a lot of ground in regards to the cross platform performance on some of the better known DAW's in the market, but without doubt the one DAW that is more synonymous with the whole area of DAW's than any other is AVID ( formally Digidesign ) Protools. Where Protools was significantly different to the others was its genealogy was not based in Native DAW's, but in hardware specific DSP powered systems that have pretty much ruled the roost at the top end for over a decade. Their Native forays have been very restrictive in regards to not only qualified hardware , but also in regards to the features available in the lower tiered versions in comparison to the big brother TDM / DSP based systems - i.e. , track counts, no ADC, limited number of I/O and numerous other features more specific to Post/Video.

With all of the above , its been difficult to do head to head comparatives against the other DAW's for many years, but all that changed dramatically with the release of PT9 which for the first time allowed any 3rd party ASIO/Core Audio hardware to be used . This report was pretty much completed just as PT9 was released , so it is is still based on the proprietary hardware, but I have included all of the PT9 results so that there is a direct comparative with the outgoing PT8 line. I'll tackle the new open hardware paradigm in Part V:

While test system spec remained unchanged, the actual audio hardware was completely different to the previous testing due to the specific hardware requirements of the AVID DAW applications.

This was already highlighting the difficulties in trying to do head to head comparative testing due to the restrictive nature of the Protools Native platform , when even within its own product line , there were 2 very distinctly different versions that used their own specific hardware , add to that the fact that the native driver protocols were not the ones used by other DAW's - ASIO/Core Audio, but there own proprietary DAE variant.

There are respective ASIO drivers for the hardware, but the consistency was not there between the 2 , M-Audio having a far better ASIO low latency implementation on Windows for example , so testing against the other applications would have to be done with the M-Audio only.

To illustrate the state of the ASIO drivers for the Digi 002 in Win7 x64 , these are the buffer settings and reported latencies - 064 : 10.8ms , 128: 13.7ms , 256: 19.5ms, which are essential at least 3 times where they should be.

The 002 was only used in the preliminary testing to compare PTLE v PTMP and then was duly retired.

The performance of LE v MP cross platform was similar, LE being a touch better across the board , but my suspicion is that it is due to some extra padding in the buffers as mentioned before . Overall the variable in the results were within a few % , so I was comfortable in going ahead with just the M-Audio from there on.

With the choice of reference hardware sorted , I knuckled down and ran thru the test sessions on the selection of the competing DAW's with the new M-Audio Hardware. Due to the fact that I needed to retest all of the other DAW's as none of the previous testing on the RME HDSPe could be used, I took the opportunity to shift the Steinberg testing back to Cubase 5.5 , which is a closer head to head market wise, and updated StudiOne to 1.6, as the developers had contacted me to inform me that some under bonnet multiprocessor work had been done since the version 1.5 that I had tested previously

Just as I was finishing up my testing , AVID released PT9 which works with not only all Digi and M-Audio hardware , but for the first time allowed the use of 3rd party hardware running under ASIO and Core Audio

Hell had officially frozen over , and to add a final cherry, they had included ADC , the lack of on the previous versions being a major gripe.

Perfect timing , as I managed to get an upgrade online and downloaded within hours of it being available and allowed me to place a nice finishing touch on this round of testing.

To be able to use my reference RME hardware at long last within Protools , will greatly simplify the comparative testing in future, and will also have a far higher consistency. That will be highlighted in Part V when we move to the ASIO only testing , this round will be the DAE used with the M-Audio interfaces in PT v ASIO in the other DAW hosts.

DAW Benchmarks:
DAWbench DSP.

DAW Application Details :
AVID Protools MP 8.0.X
AVID Protools 9.0.x
Presonus StudioOne : Version 1.6.x
Steinberg Cubase : Version 5.5.x

Reference System Detail:
Intel i7 920 Quadcore/ 2.66 GHZ/
Intel X58 / 6 GB DDR3-PC10600.

Audio Hardware Detail:
M-Audio : Profire 610 : Driver 5082

O.S Detail:
Windows 7 x64 / OSX 10.6.2

As I mentioned in the previous report, the Channel Strip Pro was one of the newly introduced reference plugin's for DAWbench, so this was the first time I had tested it in the RTAS format cross platform.

Lets take a look at the respective results.

Protools MP 8.0 : @ 064 performed better on Windows by 47% , @ 128 performance was better on Windows by 27%, @ 256 performed better on Windows by 21%

Protools 9.0 : @064 performed better on Windows by 8%, @ 128 performance was better on Windows by 8%, @ 256 performed better on Windows by 3%

Cubase 5.5 : @ 064 performed better on Windows by 53% , @ 128 performed better on Windows by 36%, @256 performed better on Windows by 32%

StudioOne 1.6 : @064 performed better on Windows by 84% , @ 128 performed better on Windows by 83% , @256 performed better on Windows by 74%

On Windows the results were again very consistent and close to par across all of the applications , Cubase having a few more legs @ 128/256. PT8/9 RTAS again scaled very well against the VST driven hosts . Performance of PT8/9 was again identical.

On OSX, the results this time for PT8/9 were a lot more consistent , no strange anomaly's @ 064 as experienced on the MD5 , which was a relief on one hand , but only deepened the mystery further as to what actually caused it.

The performance of PT8 and Cubase was very close to par on this session, StudioOne remaining consistently behind , but where PT9 on the last session was close to PT8 , on this session it showed quite a substantial improvement , and clearly out performed all of the other DAW's.

On the previous round of testing, all of the tested DAW's did not scale as well on this particular session in OSX, and that had remained consistent in all of the testing up to PT9 on this round. That maybe hinting at some possible optimization changes under the bonnet of PT9 that has proved beneficial with the specific processing required by reference plugin.

The significant difference on this round of testing is that the cross platform performance variable for Protools is more noticeable , and leaning towards Windows, tho it should be noted that the variable is greatly reduced with Protools 9 over Protools 8.

The result @ 256 , which leaned measurably towards OSX with the MD5, has not been maintained , so some questions arise there as to whether the MD5 results are the exception or the rule.

The MD5 did perform better comparably cross platform on the last round of testing as well , so there is some consistency being maintained in what we are experiencing with the RTAS versions.

My feeling to this point is that the WaveArts plugin's are very well coded and optimised for OSX , not to say they are not well coded and optimised for Windows, but they do consistently perform better than the other plugin's in the sessions tested so far.

On to the next...

The Elysia mPressor is the only plugin in the test pool that has an inherent delay, and with Protools MP 8 not being able to automatically adjust for that , the playing field was a little uneven in regards to whether the ADC/PDC calculations would play a part in the comparative scaling against the other host DAW's. With the release of PT9 we were able to not only test the scaling effects of ADC, if any between PT8/PT9 , but also level the playing field in regards to how each DAW host arbitrates the ADC/PDC.

First off, the respective results across the 4 DAW applications

Protools MP 8.0 : @ 064 performed better on Windows by 25% , @ 128 performance was better on Windows by 22%, @ 256 performed better on Windows by 13%

Protools 9.0 : @064 performed better on Windows by 12%, @ 128 performance was better on Windows by 3%, @ 256 performed better on Windows by 3%

Cubase 5.5 : @ 064 performed better on Windows by 76% , @ 128 performed better on Windows by 59%, @256 performed better on Windows by 42%

StudioOne 1.6 : @064 performed better on Windows by 98% , @ 128 performed better on Windows by 93% , @256 performed better on Windows by 95%

The Windows results showed a measurable advantage to the VST powered application on this specific session across the board. The results for both PTMP 8 and PT9 were again identical, but what is interesting there is that the inclusion of ADC in PT9 has not had any measurable effect to scaling. That in itself is quite interesting and a definite positive for PT9.

One significant thing in regards to ADC in PT9, is how it is actually applied. Whereas in the other DAW's the PDC/ADC is applied in real time when the plugin's are activated, causing a noticeable glitch/pause as the hosts have to do the recalcs in real time, PT9 does not glitch at all when the plugin's are activated, which I have had confirmed is because the actual ADC calc's are not done in real time, instead are applied when the engine is stopped and restarted. Not a huge issue in Real World application, but something to be aware of.

On OSX the results were more consistent to the last session, no anomaly at 064 for PTMP8/PT9 as with the MD5, so that was definitely the exception not the rule , also the improved scaling of PT9 over PTMP8 has been maintained despite the inclusion of ADC , which bodes well for the ADC implementation on that platform as well.

Both PTMP8/PT9 showed the other DAW's a clean set of heels again.

An analysis of the results maintains the variable we witnessed with the CSP between OSX and Windows 7 , with it being consistently in favor of the later.

It has to be noted tho that PT9 again excelled in this session on OSX, the variable cross platform being very small, especially in comparison to the VST driven app's where it is noticeably wider. I questioned last time that perhaps being specific to the inherent delays causing a harder hit on OSX in those respective DAW's. ADC on PT9 does not seem to be introducing any added overhead , despite it being stated in the PT9 manual that it would/could, especially at its highest setting, which I had used as default, go figure.

That's not to say that other plugin's with inherent latency will not take a resource hit when using ADC, but it was not the case in this instance.

The results with this particular plugin definitely showed performance in favor of VST2 on Windows over RTAS, much more so than any of the previous, but of course that could simply mean that it is better optimised than the RTAS variant , whereas the opposite could be said for the plugin on OSX , where the RTAS version performs measurably better .

Nothing conclusive either way , but something to keep in mind as the test pool widens.

This round did add some extra variables into the mix , RTAS v VST, DAE v ASIO/Core Audio , so it was a bit harder to do direct head to head comparatives than on previous test sessions simply because of the grey areas those variables introduced .

Some of the questions raised were, was DAE equally optimised cross platform , were the plugin formats of RTAS / VST2 equally optimised even within the respective O.S platforms , were the actual latency values for the respective buffer settings within DAE/ASIO/CoreAudio consistent within an acceptable range, etc, etc.

For many years it was simply a given amongst the audio community that RTAS as a plugin format was not as efficient as VST/AU , and also that DAE did not scale as well as ASIO / CoreAudio at lower latencies.

Now I do have to note that I do not have the exact latency values for DAE in respect to the buffer settings , there could well be some added padding there , but within the available testing parameters - it did hold its own against ASIO/Core Audio.

The second significant factor was that DAE actually performed very well cross platform comparatively against ASIO/CoreAudio and thirdly and most importantly in respect to the previous reports, the cross platform performance between OSX and Windows was a lot closer than the other DAW hosts.

Of course that opens up a whole host of new questions, is it that RTAS as a plugin format is more consistent cross platform , does the same apply to DAE over ASIO/CoreAudio , is it simply that PT is coded equally cross platform while the the other DAW's are all equally badly coded for OSX, etc, etc.

I'll try and tackle at least one of those questions in the final chapter where I will ditch the proprietary M-Audio Hardware / DAE, and use my reference RME HDSPe / ASIO/ CoreAudio in PT9

Vin Curigliano
AAVIM Technology
November 14 2010

Part I | II | III | V | VI