I will not settle for average and I hope you will not either. Average will get you down the road a bit, but it's not the lasting relationship you are looking for. That's one of the reasons dating can be difficult. We don't want to be alone so we might go on a lot of average dates.
It is time to evaluate all of the relationships in your life and set boundaries and priorities with each of them. Some people will need to be removed.
In a relationship that is not quite right, we (I am) are constantly on guard for things to begin going wrong. And either you work through those issues together or you don't. And if you don't you will be miserable trying to stay together. Better to move on.