Olive tree

Hi

ive got an olive tree that's about 10 years old. It's outside in a large container and is doing well.

However, the trunk is disproportionately thin in relation to its height and spread. I've used a stake to support it. 

Id like to know if there is anything I can do to thicken up the trunk. It's full of flowers - should I remove them so the tree can focus where I want it to.

thanks for reading

regards

mark