Schizophrenic garden soil

My garden soil is very strange. I live on the South Downs National Park which is chalk but I'm very close to the river that runs down to Cuckmere Haven.
The back garden is heavy clay presumably from the river bed over the years and azaleas, camellias and all clay plants grow happily in there. I had a hydrangea which turned blue so I assume it's acidic.
We recently dug up the paving at the front of the house as we want a garden there not a concreted/paved area as we have plenty of road parking in this area. Anyway we started digging up the area which faces the downs and it is full of big lumps of chalk but it has really heavy untreated clay in it too. It seems to be both so I have no idea what I am going to do with it.
I assume you can get plants that grow happily in both clay and chalk?
Posts
Most of central Suffolk is a mixture of clay and chalk - some of the most productive and high value agricultural land in the country - just apply lots and lots of manure and get the worms working - as well as almost everything else being perfectly happy there, roses will love it
Gardening in Central Norfolk on improved gritty moraine over chalk ... free-draining.
That is great news, I have my eye on several Munstead Wood roses - in fact it would be positively rude not to plant them
Thanks.