Another way of thinking about this: Do we need a specific, unique word for the way effeminate cis men are mistreated under patriarchy that pretends it has nothing to do with misogyny? Would such a word further or hinder the goals of feminism?
Because yes, effeminate cis men are mistreated under patriarchy, and their experience is different from the experience of cis women who are mistreated under patriarchy, but this is not some unique form of oppression that is unrelated to how women are oppressed that we simply can't explain in any existing terms. It's actually quite easy to explain in existing terms: Effeminate cis men are mistreated because they are seen as more womanlike and therefore inferior to masculine men. The root problem is just misogyny. To label it as "misandry" or any other such term because the subject is a man is to deliberately obfuscate what is happening and why it is happening, and it insidiously downplays the prevalence and severity of misogyny within our society.
When a trans man is mistreated on the grounds of being trans, that is simply transphobia. When a trans man is mistreated on the grounds of being perceived as a woman or as womanlike, that is misogyny and usually also transphobia. The experience of a trans man being mistreated is different from the experience of a trans woman or a cis woman being mistreated, but in this sense, the mechanisms and root issues are the same, much like in our effeminate cis man example.
The reason transmisogyny is a term that exists and makes sense is because there are vectors of oppression that we face that trans men and cis women simply do not face. Trans men and cis women are not seen as biologically "abuser-coded" interlopers who are inherently a danger to the lives of everyone in their immediate vicinity. Trans men and cis women are, at worst, treated as women. Trans women, however, are treated either as men or as women, based primarily on what would be more harmful to us at any given moment. We are subject to (nearly) every part of misogyny, but we are also denied inclusion in the category of "woman" and painted as threats, predators, and abusers due to our perceived maleness, and we are routinely pushed out of so-called women's spaces and into our own spaces (which are typically much smaller and have basically no visibility or resources) as a result.
A good example we can look at is domestic abuse shelters. Trans people, especially trans women, are at especially high risk of facing domestic abuse. However, the vast majority of women's shelters turn away transfems where many of them will accept transmascs. This is a massive problem because it is taking resources away from some of the most vulnerable people in our society and forces them to choose between male-dominated spaces which are generally very dangerous for them or not seeking any help at all.
Trans men are not subject to this kind of treatment, the kind of constant, systematic isolation (including from queer communities) and deplatforming and pedojacketing and Kiwifarms-style harassment campaigns from within the queer community that trans women have to put up with. Trans men do not need to worry nearly as much about whether or not there will be any resources to help them anywhere when they are abused. Trans men are not frequently made to feel terrified to interact with absolutely anyone who isn't another transmasc in the way that trans women are frequently made to feel terrified to interact with anyone who isn't another transfem.
You will also notice that while transmisogyny is a term that is used to describe the unique ways in which trans women are oppressed by all levels of society that other groups aren't subject to, the idea of transmisandry exists first and foremost as a cudgel against trans women. That is, at least, the only context this toy has ever seen the word crop up in. It has never once seen a post where a trans man has talked about experiencing transmisandry at the hands of a cis person, but it's seen hundreds of accusations of trans women being transmisandrists.
The takeaway in all of this is that "transmisandry/transandrophobia" is, at best, a smokescreen that serves to downplay the severity of misogyny and transphobia and misrepresents why and how trans men are oppressed, and at worst, rhetoric intentionally weaponized against trans women to portray us as mean, hysterical bitches who need to shut the fuck up.