The company is also selling its 20% stake in south of England broadcaster Meridian to Granada as part of the deal.
The tie-up means that Carlton will broadcast to 26m people, or half of the UK population.
HTV, which covers Wales and the West of England, broadcasts to 4.5m people.
Granada was obliged to sell HTV under the terms of its takeover of United News and Media's television assets in July, which also included Anglia and the majority of Meridian.
The two companies - Granada and Carlton - now dominate the ITV industry. Carlton had originally been a rival bidder for United News and Media itself.
Carlton said in a statement:
"This
move adds to our strong position in ITV and represents good value for our
shareholders. Carlton and Granada Media are now better placed to work in
partnership to maximise the value of both ITV and ONdigital, our jointly owned
digital terrestrial television business".
Carlton is paying £181m for HTV, which made profits of £14m on advertising revenues of £110m in 1999.
It will now control 40% of the ITV advertising revenue.
However Granada will retain the HTV's main production business, which is particularly strong in children's and nature programming, and incorporate it into its own operations.
Expansion plans
Granada owns its own franchises in the Northwest of England, as well as London Weekend Television, Yorkshire TyneTees, and Meridian and Anglia in the south of England.
Carlton has the weekday London franchise as well as the Central Television franchise in the Midlands and West Country TV.
Granada has emerged in the stronger position after its £1.75bn bid for United News and Media beat out a rival bid from Carlton in July.
Granada was required to sell HTV within six months because of competition concerns.
However, the government has been encouraging the consolidation of ITV in order to strengthen it against international competition in the television and media industry, and is believed to eventually favour the creation of one ITV company - a change which would require legislation.
Both companies' share prices rose on the news.
Granada Media shares rose by 13p to 550p, while Carlton was up 15p to 400p.
"This is a good deal for both companies because you're consolidating ownership between the two companies and it didn't go to a third party or a smaller group," said Simon Wallis, media analyst at WestLB Panmure.