As a former passive-aggressive introvert, I will tell you that this is a bad assumption.
I grew up in an environment where my Mother was very direct in a way that showed a lack of empathy. As a child, that embarrassed me and made me feel bad for others. My father was passive-aggressive until his drinking made him bold. Then it was all anger expressed inappropriately (shouting and cursing).
Consider yourself fortunate if you had parenting that taught you how to be empathetic and how to communicate your feelings appropriately. I had to learn all of those skills as an adult.
I once mentioned my former passive-aggressive tendencies to someone and they refused to believe that I could ever be that way. I now acknowledge the deeply empathic capabilities I possess as a result of growing up with that passive-aggressive father. So, I will say that there is hope that anyone can change. But it takes a wicked determination to break out of dysfunctional patterns of being.
I am glad for those who were patient and understanding while I worked my way through it.
