Limp mode. Next time when it does this, turn the key off and then back on, if it fixes the issue for a few minutes or moments, you are going into limp mode. Limp mode is triggered by over boosting for a period of time (about 5-10 seconds). This is the most common issue. its usually caused by a bad vac (boost) line to something that causes the waste gate actuator to not work properly when commanded.
Common issues and the fix for them.
Vacuum lines or boost lines: Inspect or replace them all. usually 3mm, 4mm and 5mm hoses. dont use cheep ones, get quality silicone or VW braided lines from IDparts.com or where ever. Not sure about your ECU but on MK3's there is a boost line going into the ecu and inside the ecu to the MAP sensor and it needs replacing too, if you have one.
N75 Valve: you should have a slight clicking from it, kinda like a CCV valve. some times you can clean them, they are not very cheep ($75) for an OEM. usually replacing it is best.
MAF: this is the sensor on the intake air duct right after the filter. ONLY CLEAN WITH MAF cleaner. A cold air intake usually is the culprit for killing them. Check out this link to understand why. Disconnect it and if driving conditions do not improve then its bad. (only after you make sure you have no boost leaks and all your lines are perfect shape.)
http://forums.tdiclub.com/showthread.php?t=290373
Boost leaks: Check all your pipes after the turbo to and from the intercooler and to the intake manifold. take them apart, and put back together to make sure no cracks etc. inspect the intercooler too. You will see some oil on the pipes. Its normal. Just wire off the parts your connecting with a light amount of brake clean on a rag. DO NOT spray it into or on the pipe. it could cause a temporary runaway. Happend to me lol, i was lucky, kinda.
thats about it. Fix and inspect those items. in that order one at a time with test drive in between to make sure you dont jsut thorw parts at it.
From what you described, Vac lines, turbo actuator, and you have a boost leak, BAD too, your IC pipes and boots, check them