You know -- I think that we have a really absurd view on sex chromosomes and how they pertain to physical and emotional characteristics. Many more people are intersex than transgender in the USA (as far as we know, (closet))
Anywho, given that more people are intersex than transgender (and most don't even know it,) it is highly likely that many transgender people are likely intersex.
However, this also assumes that many of these characteristics are derived from the sex chromosomes.
A bit about me real quick before we go talking about genomics. I am an applied scientist, although I have practiced a reasonable amount in the domain of genomics, this is not necessarily my domain. That being said, I am not an expert, but certainly do have enough experience to know a thing or two about genetics.
Our view on sex chromosomes is completely absurd. The idea that 2 (sometimes 3 ;)) chromosomes determine the way you want to dress, behave, be treated, and present yourself is completely absurd when you consider that we have over 22,000 of them. Additionally, chromosomes are capable of carrying anywhere from 50 to 300 MB of data.
We meet in the middle with 200 MB, and multiply 200 * 22,000
4400000 MB / 1024 = 4296 GB
That's almost 5 terabytes of data, and you mean to tell me that only 0.00009090909 percent of that determines your entire personality?
The reason I point this out is solely because what you say might be true for a lot of trans people, but does not have to be the precedent. This could open up the door for something like " your identity isn't valid because you failed the intersex test" which is also a wrong way of thinking.
Your proposal makes sense, and I agree it is very well possible.
My proposal would be that there is a very high likelihood that many of these genetic variations are vestigal. As humans, we have 4 vestigal toes on each foot and several vestigal organs. This is extremely common in Biology for various reasons. My thought is that there are likely a lot more genes than just the sex chromosomes that have to do with what we refer to as " biological sex." We do not fully understand our genetics, and it is most likely true that at some point in our evolution our species was either asexual (biological not socialogical) like many microbes such as Yersinia Pestis or intersex like many animals such as snails... Or both. Such things could also be carried from breeding with other hominids such as Denosovans or Homo Erectus.
It is entirely possible that transgender people carry these vestigal genetics, regardless of the two " known" sex genes that seem to cause variations to our hips, hormones, and genitalia development.
I would love to do a bit of testing on this, but of course to do so would require some DNA samples from various intersex, transgender, and cisgender people -- and even with that, 22,000 is a LOT of observations, that each unpack into a lot of data to observe -- hinting at why we really don't have a firm understanding of this at all.