They argue that the total volume of water in aquifers underground is 100 times the amount found on the surface.
The team have produced the most detailed map yet of the scale and potential of this hidden resource.
Writing in the journal Environmental Research Letters, they stress that large scale drilling might not be the best way of increasing water supplies.
Across Africa more than 300 million people are said not to have access to safe drinking water.
Demand for water is set to grow markedly in coming decades due to population growth and the need for irrigation to grow crops.
Freshwater rivers and lakes are subject to seasonal floods and droughts that can limit their availability for people and for agriculture. At present only 5% of arable land is irrigated.
Continue reading the main story
What is ground water?
When water falls as rain or snow, much of it either flows into rivers or is used to provide moisture to plants and crops. What is left over trickles down to the layers of rock that sit beneath the soil.
And just like a giant sponge, this ground water is held in the spaces between the rocks and in the tiny inter-connected spaces between individual grains in a rock like sandstone.
These bodies of wet rock are referred to as aquifers. Ground water does not sit still in the aquifer but is pushed and pulled by gravity and the weight of water above it.
The movement of the water through the aquifer removes many impurities and it is often cleaner than water on the surface.
Now scientists have for the first time been able to carry out a continent-wide analysis of the water that is hidden under the surface in aquifers. Researchers from the British Geological Survey and University College London (UCL) have mapped in detail the amount and potential yield of this groundwater resource across the continent.
Helen Bonsor from the BGS is one of the authors of the paper. She says that up until now groundwater was out of sight and out of mind. She hopes the new maps will open people's eyes to the potential.
"Where there's greatest ground water storage is in northern Africa, in the large sedimentary basins, in Libya, Algeria and Chad," she said.
"The amount of storage in those basins is equivalent to 75m thickness of water across that area - it's a huge amount."
Due to changes in climate that have turned the Sahara into a desert over centuries many of the aquifers underneath were last filled with water over 5,000 years ago.
The scientists collated their information from existing hydro-geological maps from national governments as well as 283 aquifer studies.
The researchers say their new maps indicate that many countries currently designated as "water scarce" have substantial groundwater reserves.
However, the scientists are cautious about the best way of accessing these hidden resources. They suggest that widespread drilling of large boreholes might not work.
Dr Alan MacDonald of the BGS, lead author of the study, told the BBC: "High-yielding boreholes should not be developed without a thorough understanding of the local groundwater conditions.
"Appropriately sited and developed boreholes for low yielding rural water supply and hand pumps are likely to be successful."
With many aquifers not being filled due to a lack of rain, the scientists are worried that large-scale borehole developments could rapidly deplete the resource.