"Losing Jermain made it difficult, but we kept the ball well," Redknapp told BBC Sport.
"At half-time I said to the lads I felt there was another goal for us. In the second half they pressed us better but we hit them with a fantastic goal."
The three points ensured the north London outfit maintained the pressure on the Premier League's top four as they cemented their place fifth in the table.
Dutch midfielder Van der Vaart got Spurs off to a flying start when he finished off a stunning counter-attack before his team were reduced to 10 when Defoe was sent off for the use of a leading arm when jumping with Collins.
However Redknapp was not convinced the dismissal of the striker was a correct decision.
"It is not in the little fellow's nature to do that. I haven't seen a replay and I said at the time he must have caught him but a few of the lads have looked at it at half-time and felt it was a very harsh decision.
"The little fellow said there was no way he meant to catch him with his elbow and he was very unlucky I felt."
Despite their numerical advantage, Villa were restricted by a resolute defensive display with Gabriel Agbonlahor and Carlos Cuellar wasting the few chances the hosts did create.
Van der Vaart made sure of the points with a second strike and although Albrighton pulled one back with eight minutes left, Redknapp's men were left celebrating a crucial victory.
The Dutchman, recently recovered from a hamstring injury, was instrumental in the victory and took his tally of goals for the season to double figures in the process.
Villa deserved a point - Houllier
"Van der Vaart is a decent player among a lot of good players," added Redknapp.
"His goalscoring record is amazing and he is capable of getting 15 to 20 goals a year because he is such a fantastic finisher. His all-round game is fantastic."
He added: "When you can win away with 10 men I think we did a great job. The second one was really important and then I had the feeling we would win the game"
Meanwhile, Villa boss Gerard Houllier remained upbeat despite the defeat leaving his side hovering dangerously four points above the relegation zone.
"I think it is a poor result, but not a poor performance," the Frenchman told BBC Sport.
"The mistake was made was conceding a second goal, after that we knew it was difficult. We got a goal and probably deserved an equaliser which we didn't get.
"We played better in the second half but the first goal killed us a bit as until then it was equal but you have to give credit to their performance.
"It is a shame we didn't manage to equalise but we have to blame ourselves for the mistakes we made."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.