Mulberry Tree fruit

I planted a Mulberry tree about 5 years ago. Every year it grows wonderful leaves. The problem is the fruit. They are TINY (only about 1.5cm long). I have watered the tree and fed it. I’m not sure of its variety but I vaguely remember the tag saying it was a white mulberry tree. I was shocked when there were no white parts of it as the fruit starts red and turns black. I barely notice the white flowers. They come and go so fast and the tree is at the side of the property. It gets full sun, has good drainage and has heaps of room to grow as much as it wants to. It’s about 4 meters tall now. Does anyone have a clue as to why the fruit is so small? The fruit tastes wonderful but I can barely pick it.