Ferguson also felt Didier Drogba pulled Brown's shirt as Terry's glanced header flew into the corner.
"That goal should never have been allowed," added Ferguson. "We've dominated the game. We've had great chances to win the match and not taken them - and that's our fault.
"We had great opportunities to get to the edge of the box and some really good chances in and around the box, but we should be finishing it off.
"We've only ourselves to blame in that respect. But you do need a break - and we never got the break we needed."
Whilst Ferguson has escaped punishment from English football's governing body, Wayne Rooney has been warned about his future conduct by the FA.
The England striker was booked during the game for protesting Chelsea's goal and when leaving the pitch at full time mouthed the words "12 men" into a television camera.
Midfielder Fletcher was adamant that Atkinson made the wrong decision in penalising him for his challenge on Cole.
"I've watched it again and it's clearly not a foul," said Fletcher. "Ashley Cole jumps up in the air and the referee gives it."
Phelan 'proud' of Man Utd display
The Scotland international also expressed frustration at the special treatment afforded to officials when it comes to negative judgement on their performance.
"Referees are the only people in the game you can't criticise," he added. "Managers and players can be criticised but you cant criticise referees."
United assistant manager Mike Phelan echoed the comments of Ferguson and Fletcher.
"First and foremost we didn't really think it was a free-kick," said Phelan. "Fletcher was committed to the tackle, he won the ball and Cole was just getting out of the way of it.
"On the goal itself we felt there was an infringement there on Wes Brown but the referee wasn't in a very good position to see it.
"The manager talks about referees but I think sometimes he has a point. He has to make these points to try and improve the standard."
Phelan, though, refused to blame the United players for their defeat, instead suggesting that their performance warranted greater reward.
"Tactically we did very well," he added. "The players stuck to their tasks but sometimes you don't get what you deserve.
"I think Chelsea realise they were in a game today."
Ancelotti hails 'important' victory
Ferguson admitted the defeat - United's third on the road this season - was more significant than their recent 2-0 reverse at Liverpool.
"The fact is Chelsea go five points clear of us," he said. "The defeat to Liverpool didn't affect our position in the league. If we'd have won today we'd have gone top."
Ferguson's counterpart Carlo Ancelotti, meanwhile, refused to be drawn on the legality of the winner, saying only: "It is difficult for me to say until I have seen it again, but I do think it was a foul [on Cole]."
However, the Italian was keen to play down suggestions the victory gives his side a big advantage in the title race.
"It was significant to send United five points behind us - this is important - and it means we have won a game against one of the most important teams in England," he said.
"But we must retain our concentration. I hope the players return from the international break in good condition and that we continue as we are.
"We've only played 12 games and we know very well the season is long and we have to maintain our good condition, a good mind and good performances.
"Today was a little step [towards the title] but it's a long season."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.