Eyewitness reports suggested opposing fans threw bottles and bricks at each other and starting small fires as police moved in to try and quell the disturbances.
Inside the stadium, police and stewards battled to remove fans from the pitch throughout the match, which West Ham won 3-1 in extra time.
The first pitch invasion was after Junior Stanislas' equaliser, while the second more serious encroachment occurred after the same player slotted a penalty to give the Hammers the lead.
Play was briefly suspended when referee Paul Taylor decided to take the players off the pitch.
It took officials about six minutes to clear about 50 people from the pitch and police had to redeploy within the stadium to tackle the invasion.
After the match, riot police lined the streets outside the ground, roads were closed and a helicopter flew overhead in preparation for further possible clashes as thousands of fans tried to leave the stadium at full time.
Hammers boss Gianfranco Zola condemned those responsible for the trouble after the match and said they were not fans of football.
"I've played seven years in England and I've never seen it like this," he said.
"It's certainly not good for football, I was completely shocked. Totally. I knew it was a game that meant a lot for the two sets of supporters but I didn't imagine it like this.
"What can I say? I'm a sport man. I love the game. I love to go on the pitch and try to make it exciting for the supporters and enjoyable for everybody to watch. This was beyond my powers."
Millwall boss Kenny Jackett said: "I didn't fear for my own safety. There were no Millwall fans on the pitch, they stayed where they were supposed to."
But Millwall striker Neil Harris said: "It was extremely hostile and volatile. All my concern was for my team-mates. At one stage it wasn't very pleasant."
Sports Minister Gerry Sutcliffe gave his full support to the FA's investigation, adding: "We have made great progress in the past 20 years in tackling football hooliganism in this country and will not tolerate any return to the dark days of the 70s and 80s when it plagued the game."
The Football League, which organises the Carling Cup, condemned the scenes and promised to help identify and punish those involved.
Malcolm Clarke, chairman of the Football Supporters Federation, told the BBC he expected to discuss the violence at a routine meeting with FA chief executive Ian Watmore on Wednesday.
"It's important to get to the bottom of it but too many hysterical over-reactions don't help," he said.
"I can assure you the vast majority of Millwall and West Ham supporters will be as horrified by events that occurred as everybody else."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.