Colin Lee's side looked to be cruising to a comfortable win when Dunne (47mins) and Asaba (53mins) fired the visitors 2-0 ahead.
Millwall's cause appeared to be further assisted when Yeovil defender Liam Fontaine was shown a straight red card on 59 minutes for pulling back Barry Hayles when he was clear through on goal.
But instead of helping Lee's men, it was Yeovil who powered forward and piled on the pressure with shots, headers and crosses raining in on the visiting goal.
That pressure finally paid off four minutes from time when Aaron Davies slid home after The Lions had failed to deal with Phil Jevons' cross-shot.
It really was panic stations thereafter as Lee's side were rocking under intense pressure from the home side, often causing their own problems with poor, hurried clearances that found the opposition instead of a blue shirt.
But The Lions weathered the storm, and when Alejandro Melono smashed an injury-time shot a foot wide of Andy Marshall's post, Lee's men were home and dry.
Yeovil finished the game as they had started it, and The Lions were indebted to keeper Marshall, who produced a string of fine saves before joining his team-mates in being struck by an attack of the jitters in the latter stages.
Millwall boss Lee was forced to make four changes to the side that had recorded victories at Wolves and Sheffield Wednesday, with Adrian Serioux, Sammy Igoe, Carl Asaba and Paul Robinson coming in for the unavailable Ady Williams, Phil Ifil, Jermaine Wright and Ben May.
Not surprisingly, it took the new-look Lions a while to settle down, whilst Yeovil came flying out of the traps, going close on six minutes when Terry Skiverton headed just wide from the first of three corners in the opening half.
Pablo Bastianini, who was on trial at Millwall during pre-season, was only denied a sensational opening goal after 15 minutes when he hit a first-time shot from inside his own half which Marshall somehow tipped over the bar.
Yeovil clearly felt that their best chance of scoring would be from distance, and they peppered the Millwall goal at every opportunity with Marshall again coming to the rescue to deny the busy Bastianini and Lee Johnson.
At the other end Carl Asaba's 30th minute header was cleared off the line by Darren Way following Jody Morris' corner on the left.
But it was the home side who looked more likely to score with Skiverton twice enjoying the freedom of the Millwall defence to pick his spot from corners, only to head wide of the mark on each occasion.
Gary Johnson's men finished the half with yet another long-range effort, this time johnson shooting straight at Marshall from 20 yards as Yeovil continued to tease and torment The Lions' rearguard.
The half-time whistle at least gave Den boss Lee an opportunity to take stock and address the defensive frailties that had given Yeovil hope of pulling off a Cup shock, beefing up the midfield with the introduction of Don Hutchison for Igoe.
Within four minutes of the restart Dunne had made the breakthrough, lashing home from Hayles' tantalising left-wing cross.
On 53 minutes the game looked to be safe when Asaba rose to meet Robinson's hopeful forward punt, and outjumped keeper Chris Weale before steering home his second goal in successive games.
Hutchison then fired wide with a fierce freekick and Serioux's cross clipped the bar as The Lions sensed further goals were in the offing.
But the dismissal of Fontaine just before the hour strangely seemed to lift the home side, who sensing all was not lost, re-grouped and produced wave after wave of attack on the Millwall goal.
The busy Marshall twice blocked efforts from Jeavons, tipped over Matthew Harrold's header and then held Davies' 20-yard drive.
But there was little he could to keep out sub Davies' close-range effort after 86minutes to set up a nailbiting finale in which The Lions somehow held out for a third successive away victory.
Yeovil Town: Weale, Skiverton, Way, Johnson, Gall (Harrold, 57mins), Jevons, Jones (Sodje, 60mins), Amankwaah, Bastianini (Davies, 66mins), Fontaine, Melono. Subs not used: Terry, Collis.
Sending off: Fontaine (59mins)
Scorer: Davies (86mins)
Millwall: Marshall, Dunne, Vincent, Morris (Craig, 71mins), Phillips, P.Robinson, Livermore, Serioux, Igoe (Hutchison, 46mins), Asaba (Fangueiro, 78mins), Hayles. Subs not used: Jones, Braniff.
Booking: Vincent (64mins)
Scorers: Dunne (47mins), Asaba (52mins)
Ref: M. Russell
Att: 5,108
