An army officer said on state radio a "consultative council" of civilian and military chiefs would be set up. The EU and African Union condemned the move.
googletag.cmd.push(function() { googletag.display('content1'); });
All ministers and other top officials have been summoned to the main military camp "to guarantee their security".
But Guinea Prime Minister Ahmed Tidiane Souare said the government "continues to function as it should".
googletag.cmd.push(function() { googletag.display('content2'); });
And National Assembly Speaker Aboubacar Sompare, the constitutional successor, told French television there had been an attempted rebellion but he did not think all the military was behind it.
'Long illness'
He announced earlier that President Conte, who ruled the West African country with an iron fist for 24 years, had died after a "long illness". Forty days of national mourning have been declared.
The precise circumstances of his death are unknown, but Mr Conte was a chain-smoker and diabetic. It is also believed he had been suffering from leukaemia.
BBC West Africa correspondent Will Ross says it is important to see whether the army is united on the way forward for Guinea, as a power struggle could be extremely dangerous given the deep ethnic divisions there.
Guinea's neighbours - Liberia, Sierra Leone and Ivory Coast - are enjoying relative stability after years of conflict and there are fears any unrest there could spread and embroil the sub-region in fighting once more.
'Stay at home'
Only hours after the announcement of the president's death, a junior army officer went on state radio to say the army had taken over, and a body called the National Council for Democracy and Development (CNDD) set up.
"As of today, the constitution is suspended as well as political and union activity," said Capt Moussa Dadis Camara. "The government and the institutions of the republic have been dissolved."
Capt Camara, who is head of the army's fuel supplies unit, said an interim council would be set up to root out corruption and organise fair elections.
Announcers said the officer was speaking on behalf of the entire military, although this has not been independently confirmed.
A later statement by the CNDD told ministers to present themselves at the Alpha Yaya Diallo military camp "to guarantee their security". It also ordered people not to loot and to stay at home.
"Public assemblies are formally forbidden," it added.
African Union peace and security commissioner Ramtane Lamamra told the AFP news agency: "If the army coup is confirmed, it is a flagrant violation of the constitution and of African legality."
Former colonial power France - in its capacity as the current holder of the European Union's rotating six-month presidency - said it would oppose any attempted putsch in Guinea and called for free and transparent elections.
The BBC's Alhassan Sillah in the capital, Conakry, said soldiers have set up check-points along the main roads into the city centre, but so far there have been no reports of them being heavy-handed.
Vehicles checked
Vehicles are checked briefly and waved through, he says.
We need a change, change that will benefit all Guineans. We pray for a good leader Amara, Nzerekore
Earlier, the leader of the Union for the Progress of Guinea and the secretary of the opposition alliance, Frad, Jean-Marie Dore, called for a peaceful transition of power.
Veteran opposition leader Alpha Konde returned to Guinea on Sunday after 15 months of self-imposed exile in France. He left Guinea after being released from jail.
According to the constitution the National Assembly speaker should be in charge until a presidential election is held within 60 days.
The BBC's Will Ross says many analysts had predicted the army would try to take over following Mr Conte's death because he had been increasingly relying on it to shore up his oppressive rule.
General Conte came to power in 1984 at the head of a military coup to fill the vacuum left by the sudden death of his predecessor, Sekou Toure, who had been president since independence from France in 1958.
He eventually oversaw a return to civilian rule and was elected three times, although critics said the votes were never free or fair.
As his health declined over the last five years, it was often unclear who was in charge and the government barely functioned, our correspondent says.
Although Guinea's mineral wealth makes it potentially one of Africa's richest countries, its population of about 10 million is among the poorest in the region.
Story from BBC NEWS:http://news.bbc.co.uk/go/pr/fr/-/2/hi/africa/7796902.stm
googletag.cmd.push(function() { googletag.display('comments'); });