Cross Platform DAW Performance - Part II :

In my closing comments of the last report I made a note that Steinberg were soon to be releasing Nuendo 5.0 / Cubase 5.5 , which they promised had significant improvements in low latency performance, with a specific focus being on OSX to improve the comparative performance variable between the competing operating systems.

Leading up to the release Steinberg reps had been publicly quoting numbers as high as an 80% improvement in low latency performance which was astounding to say the least , but they never really clarified the details of how those figures were achieved , nor whether that was specifically for OSX. Of course I was more than a little interested in seeing how the numbers actually panned out , whether the comparative performance between OSX and Windows which was highlighted in the first report, had been narrowed, and whether there was any improvement on the Windows side of the fence as well.

I expanded the testing to include a suite of 3rd party plugins and also have done comparative testing against StudioOne 1.5 and Reaper 3.6 in Part III to get a better cross reference.

In preparing for the new round of benching , my tried and tested X58 / i7 920 was again used to conduct the cross platform testing on both Win7 x64 and OSX 10.6.2. The configuration was identical to the first round except for using the latest version of the applications obviously.

This time I do not have the cross reference results of the Mac Pro from Pal Svennevig who helped me enormously during the first testing phase across both OSX 10.5.8 and OSX 10.6 .

His contribution on the first round cannot be understated as it brought much needed clarity to the OSX results on my system.

With the cross platform performance results of the single i7 system being qualified as accurate, I felt that the cross reference was not as important this time around.

The volume of time and energy required for the testing and cross referencing is quite extensive and Pal was unfortunately unavailable for the redux in testing the new versions.

Time constraints for both of us has been very tight over the last few months , so I'll be presenting the results on my reference system and may amend the report if I do eventually get the test results from Pal in future.

DAW Benchmarks:
Cubendo DSP / DAWbench DSP.

DAW Application Details :
Steinberg Nuendo : Version 4.3.x
Steinberg Nuendo : Version 5.0.x

Reference System Detail:
Intel i7 920 Quadcore/ 2.66 GHZ/
Intel X58 / 6 GB DDR3-PC10600.

Audio Hardware Detail:
RME : HDSPe : Driver 3.0x

O.S Detail:
Windows 7 x64 / OSX 10.6.2

You are probably reading the heading and wondering what the hell I am talking about, so some clarification is in order. For those that have not been directly involved with the Steinberg version leapfrogging paradigm over the years, they would have no idea just how convoluted the Steinberg nomenclature is. A bit of history to clear the fog.

Steinberg Nuendo and Cubase were initially very different beasts that had completely separate code bases ie. SX1.x and Nuendo 1.x. At version 2 (seq2) - Cubase SX2 and Nuendo 2's code base was unified, both applications were a huge departure from the previous generation and heralded a whole new era, where the development of both were supposed to be synchronized for the most part. Nothing particularly special there as every other DAW with a tiered structure had very similar if not identical development cycles.

Nuendo 4.0 never in fact saw the light of day IMO , there was never a Nuendo version with that first revision of seq4 , instead after an extended delay of close to 12 months Nuendo 4.1 and Cubase 4.1 were simultaneous released (seq4 rev2)

With both applications now back in sync, the hope was that the development and nomenclature would remain so in future.

That may well have been the case if the now even more infamous solo / mute bug hadn't reared its head as that delayed N5 close to 16 months over the release of Cubase 5.

In the meantime the Nuendo developers went back to the drawing board and set to work to resolve a whole bunch of bugs that had been left hanging at the planned end of seq4. By that stage the nomenclature was already out the Window as Cubase had reached 4.5 , while the same update for Nuendo received the moniker of 4.2. The reasoning being some lame spin about a non finished synchronizer feature, and that Nuendo would receive the 4.5 moniker in a later patch. Now remember they had already officially closed off on Cubase 4.x and moved to Cubase 5.x

Confused yet.. ?

First cab off the rank is the Cubendo DSP version of the test session using the bundled Steinberg Multiband Compressor that was used in the first round which is VST3. The results of N4.3 and C5.0 used in the first round were close enough on par so the transition to using Nuendo instead of Cubase was a non issue.

All results are with Hyperthreading ON, there are no results for Hyperthreading OFF nor will there be in any future testing as I believe I have enough empirical evidence that despite Steinberg's official line to the contrary, that performance is better cross platform with HT ON for the vast majority of cases.

What is most evident when analyzing the results is that despite Steinberg's bold claims of huge improvements with scaling in N5 on OSX , on this specific test session the only quantifiable improvement was in fact on Windows 7 x64, which showed a 7-10% improvement across the board , whereas on OSX the results for N5 where a touch under those of Nuendo 4.

After Steinberg's repeated claims prior to the release of huge performance gains on OSX , to say I was surprised and disappointed with the results is an understatement.

One positive is that the performance under Windows 7 had improved, but I needed to clarify whether that would be maintained once we ventured away from using the bundled plugin and used some 3rd party reference plugins that could also be used in other DAW applications.

There were also the added variables of VST3/VST2 and also inherent delays imposed by the plugins that would need the added arbitration of PDC/ADC ( Plugin Delay Compensation / Automatic Delay Compensation ) - the Steinberg MBC had 1712 samples of inherent delay.

The URS Channel Strip Pro was the first of the new 3rd party plugins that I have chosen after an extensive process of testing , retesting and evaluation across not only both platforms , but also plugin formats. It also deviates from the multiband compressors I have used in the past by allowing numerous DSP processes to be applied per load channel.

On Windows the results were pretty much on par between N4 and N5, and have remained reasonably consistent to the results I achieved with the MD5.

The results on OSX mirrored the results for the MD5 in that N4 was consistently better than N5 across all latencies.

Again not boding well for the reported OSX optimizations in N5 using a 2nd 3rd party VST plugin.

The cross platform comparative results this time did not show the reduced variable that was witnessed @ 128 / 256 on the previous MD5 round of testing , instead remaining very close to the variable that was experienced with the Steinberg MBC.

So to summarize , Windows performance between N4 and N5 was pretty much identical with only a variable of 1 plugin on 128/256 buffers - which is insignificant to say the least.

On OSX results have remained consistent across all 3 plugins tested thus far, with performance being measurably worse in N5 both with the onboard VST3 and 2 x 3rd party VST 2 plugins. This plugin also has no inherent delay , so that may rule that out as a variable.

The final 3rd party plugin used is the Elysia mPressor which is a vintage emulation styled compressor that adds a few other variables into play that were not present in the MD5 and CSP , that being an inherent delay and also a VST3 version.

In that regard this plugin has the most similarity to the bundled Steinberg MBC, so it would be interesting to see whether the cross platform variables were more consistent to that plugin or the later VST plugins.

The real time testing methodology of the benchmark is quite brutal when using plugins with inherent delays , as the PDC/ADC calculations have to be done on the fly as the session is playing back, so there is a pronounced pause/glitch anytime a plugin is activated.

This is very evident when using plugins without any inherent delay, as running thru the benchmark incrementally is very smooth in comparison.

The 2 VST plugins without the inherent delay didn't really stray too far off the initial results, only distinct difference being that on Windows 7 , there was not a measurable improvement in Nuendo 5 as was experienced with the VST 3 plugin, so the remaining variable to be explored was VST3.

I should note that all the results listed above are VST2, I did initially start the comparative VST2 v VST3 testing, but with all future cross platform testing on the other DAW app's being VST2, and the initial results not showing consistent behaviour to the Steinberg MBC, it was not looking like VST 3 being a variable in the cross platform results

If anything, the VST3 plugin did worse on Win7 at 032/064 on both Nuendo and StudioOne ( more on that in the next report ). On OSX the results were identical , so I safely ruled out VST3 as being a contributing factor and continued on with the VST2 plugin

The Windows results for N4 v N5 were close enough to par , so again there was no significant improvement in N5, on OSX we see a clear performance advantage of Nuendo 4 over Nuendo 5 , so the more 3rd plugins we test , the more Steinberg's claims of improvement on OSX are slipping.

The overall results remained very consistent in regards to the cross platform performance comparative.

As I stated in my first report, the results are in no way conclusive in regards to overall performance for each respective operating system, but for the Steinberg product the performance on Windows still remains far superior on the new versions

This round of testing initially started out as a direct comparative to explore the much publicized OSX multiprocessor / low latency scaling improvements that were supposed to be in Nuendo 5/ Cubase 5.5 , unfortunately I am drawing a blank in that department, but it did motivate me to expand the reference plugins used that will also be used as we explore the other DAW applications.

That closes off the Steinberg specific testing, the next report will cover comparative performance of 3 DAW's - Nuendo 5 , Reaper 3.6 and StudioOne 1.5 using the 3rd party plugins to get as consistent a test platform as possible. All use VST Plugins/ASIO/Core Audio, all have similar playback buffering routines, and of course , and most importantly , all are developed cross platform.

Vin Curigliano
AAVIM Technology
October 21 2010

Part I | III | IV | V | VI