Although it's easy to say, "Just be sure it says 505.01.", I do not believe that to be true.
For example, we do think the Valvoline MST Synpower in 5-40 weight is an acceptable oil for our PD cam kit, we also know that the most popular oil that we recommend is NOT 505.01 certified. Quite honestly, it is because the Schaeffer's Oil company re-evaluated the 5-40 oil and found at the 100c temperature that they perform viscosity testing, the oil was slightly lower than intended viscosity. They took the 505.01 rating OFF. Personally, I wish they had made the oil a skosh thicker..
When we have oil testing services evaluate Schaeffer's samples, the results are consistent..."Although the oil is slightly low viscosity, it apparently does not matter, as metallurgy in the oil is excellent." That is to say, the wear index is very low for our Schaeffer's.
At the time Schaeffer's took the rating off, we had many engines running on the Schaeffer's oil. When we asked Schaeffer's if when testing, they were still using the same formula. Schaeffer's response was it was exactly the same formulation. Since we had many engines exceeding expectations with our cam kit, we decided 'field testing trumps certification'.
But that is not to throw the whole 'correct oil' concept out the window. On the contrary, we find the 'approved oils' from Castrol; TXT and SLX were very prone to sludging. We also saw comparatively early cam damage, but cannot offer any compilation. It's really about the sludge that I do not recommend it.
There are some other oils we feel may be dubious. Is there a 'best'? There are certainly better and not-as-good categories.
As example, another non-505.01 oil we had a daunting experience with was Mobile 1 TDT 5-40, Our own testing seemed to indicate the Mobile 1 was a problem. Then about 4 years ago, we had a rash of issues during a very hot summer. Several Texas and Oklahoma vehicles suffered catastrophic failure of the #4 piston and cylinder. This issue was on several cars that were driven nearly exclusively at highway speed and almost all had short interval oil changes. 4000k-6000k. The only common denominator for cylinder/ piston failure between 6 engines was Mobile 1 TDT. The decision is easy enough... NO MOBILE 1 for me.
Another issue we are seeing is with the common rail engines. From the few engines we have seen torn down and the relative short mileage that some had, my thinking is that the 5-30 weight oil for the common rail should be a 5-40 oil. I think application of low viscosity oil designations have more to do with CAFE standards and compelled fuel economy, than engine longevity. Common Rail Engines that we think should still have cross-hatch in the cylinders from initial honing, are wiped smooth in engines with as little as 150,000 miles. There is another issue we see of these engines running consistently hot; 100c-105c water temps, which exacerbates the issue.
The API index is an aid to properly identify oil for it's diesel performance. What is particularly strange for the Valvoline MST Synpower is that it's rated in the API indexes as an 'SN' oil, not the CI or CJ designation for diesel engines. However, after our review from years ago and once again, recently, we can approve Valvoline MST Synpower 5-40. So it's clear, the reason for the necessary 'diesel rating' is concerning the ability to hold diesel soot in suspension to a measured saturation point. Although it is possible some of the oils mentioned here would qualify, the CJ and CI ratings test the oil for this ability.
The API must be a 'membership company', as there are other oil companies, like Redline, whose products are touted to perform certain ways, but they don't supply any API information. That leaves me thinking you become your own field testing unit.
With all that said, I strongly agree with Gofaster and his implication that data from large samples is the way to go. Field testing is the answer. And there is likely, an oil that is overall, superior. The problem is, I only have a few oils that I can honestly say we have extensive results with, and the largest sampling is the Schaeffer's. As for most of the other oils, we are really only getting sampling information when there is a engine failure. We always ask, "What oil are you using?" That has given us a 'feel' for the business, especially for the PD motors.
I might mention, the ones who own the pre-2005.5 PD's or the 06 Beetles and Golfs, they still have the BEW engine. The BEW has proven to be a longer lived engine, regardless of straying from the best engine oils. I'd rather think it would be wiser to depend on field testing over certifications. That is why we continue with our favorite oil.
What's the best oil to use for a PD engine? We think it's Schaeffer's, based on our extensive field testing. Is there something better out there? At $6.50 a quart, our everyday price? Maybe...