Clay Deposits are "mined" using a pickaxe to extract bricks of Clay. Each of the homesteads in the Hearthfire DLC has a clay deposit nearby.
Bleakwind Basin, just outside the camp's northern edge, near the bank of a pond.
Bruca's Leap Redoubt, follow the path southwest out of the main entrance to the camp. The trail terminates at the river; the deposit is on the right on the river bank. The deposit is just down the mountain and across the river from Harmugstahl.
Lost Valley Redoubt, Right below the Forsworn camp, there is a lake in front of the waterfall. The clay deposit will be directly north of the waterfall. It will be on the edge of the lake.
Nightgate Inn, three are located here on the banks of the pond behind the inn: one just underneath the dock's left-hand side, one on the southern shore (right-most bank from the dock.) and one on the eastern shore (directly across the pond from the dock.)
Riverwood, Go down the stream next to the log mill. Keep going up the stream until nearing the rapids. The clay deposit will be under a ridge to the left right before the rapids.
Robber's Gorge, across the river and to the southwest along the west bank of the river.
South Brittleshin Pass, Go south from the entrance and a short wooden pier with the end slightly sunken into the lake can be found. Turn east and follow the shore of the lake to a small cut inland to find the clay deposit.
Treva's Watch, two deposits across the river from the fortress, one more or less due west and another to the south and slightly east.
Valtheim Towers, under the bridge on the other side of the river, and another a little bit upstream.