The restaurant is near to a police station
David Hernandez, a paramedic at the scene, said three police officers and a woman had been taken to hospital with serious injuries.
It is believed the bomb had been on a bicycle.
Police defused a second bomb found in a northern Bogota neighbourhood, said police spokesman Sgt Alberto Cantillo.
He confirmed that four officers had died in the blast.
A large crowd gathered at the scene of the explosion, where a small body lay in the street covered with a sheet. Another body lay on the pavement nearby.
Prime suspects
The Josefa restaurant is across the street from the Fatima police station in the south of the city and is popular with officers.
The blast shattered windows in a neighbouring building as well as the windscreen a police truck parked in front of the restaurant.
The BBC's Jeremy McDermott, in Medellin, says the prime suspects are the Marxist rebels of the Revolutionary Armed Forces of Colombia (FARC).
In the last few days they have escalated attacks across the country, particularly against the infrastructure of Bogota hitting electricity pylons and the reservoir that provides much of the capital's water.
Last year, the FARC field marshal Jorge Briceno Suarez - better known by his alias El Mono Jojoy - promised to take their 38-year war against the state from the countryside into the cities.
It is there that the vast majority of Colombians live.
Landmark deal
The Mayor of Bogota, Antanas Mockus, called on residents to turn off their lights for three minutes on Friday to protest against the violence.
The government and FARC negotiators are trying to hammer out an agreement for a ceasefire in Colombia's 38-year civil war.
Last Sunday FARC signed a landmark deal with the government to reach a ceasefire by April 7.