The atmosphere is, as I have previously said, very large. It extends tens of kilometres above the Earth's surface, though because of rapidly decreasing density, half the total mass is contained in the first 5 km. Ocean volume is less, almost exactly half of the "normalised" (1 atmosphere pressure) atmosphere. A good illustration can be seen here, which shows the relative size of the Earth and spheres representing ocean and atmosphere.
However, while the atmosphere has a larger volume, it has a much smaller mass; the atmosphere is a mixture of gases, and the ocean is liquid water with small amounts of dissolved solids. Normal atmospheric pressure at the Earth's surface can balance a column of mercury (which is a very dense liquid metal) 760 mm high (just over 3/4 metre); this equates to a column of water 10.33 metres high. Another way to envisage this is to consider that the atmosphere exerts a pressure equal to the weight of the column of air above a given area. The pressure is 1.03325 kilograms per square centimetre; the height of a column of water weighing 1.033 kg, and therefore exerting the same pressure on 1 sq.cm is 1033 cm or 10.33 metres. The mass of the atmosphere is equivalent to a depth of just 10.33 metres of sea water. When the ocean area is taken into account (71% of the earth's surface), this equates to 14.5 metres depth of ocean.
When heat content or capacity is considered, the disparity is even larger. The specific heat (amount of heat needed to heat one gram of a substance one degree Celsius) of sea water is 3.93, the specific heat of dry air is 1.006.
So what does all this mean? It means that the heat capacity of the atmosphere is equivalent to just 14.5 x 1.006/3.93 or just 3.7 metres of ocean depth. The ocean's heat capacity is hundreds of times greater than that of the atmosphere. When he was explaining about the logic of lighter electrons orbiting the relatively massive atomic nucleus (rather than the opposite as had been claimed by some), Ernest Rutherford said “When you’ve got an elephant and a flea, you assume it’s the flea that jumps.”
When considering the internal driving factors in Earth's climate, the atmosphere is the flea, and the ocean the elephant.