The viscosity of oil suitable for an engine is generally far too thick to be ideal for the high speed bearings of a turbocharger. Turbomachinery oils typically run around 5 centistokes at 100ºC. 5W40 engine oils run around 14-15 centistokes at 100ºC. Excessively cool oil and the resultant high oil viscosity promotes unstable oil wedge development in high speed bearings. Oil whip or "whirl" (orbiting shaft motion) is caused by excessively high oil viscosity. Cold oil also contributes to bearing overheating as the high viscosity reduces oil flow through the bearing.
It is a compromise really. Cooling and the resultant thickening of the oil would most likely be detrimental to turbocharger performance and longevity, but would serve the low speed heavily loaded bearings of the engine quite well. Ideally, engine oil should be heated to obtain viscosity of less than 5 centistokes to work ideally in a turbocharger, and cooled to maintain a viscosity of 14-15 centistokes in the engine.