All of Kaz's strengths are simultaneously his greatest weaknesses.
His need for vengeance is what saved his life on the Reapers Barge and makes him keep on living. It's the cause of his resilience and determination. Without it, he would've given up and died many times over. And yet, it's what almost gets the Crows killed during the heist. He compromises everything solely because he can't handle the idea of Pekka Rollins being punished by anyone's hand but his own. Pekka would have suffered in the Ice Court, but it would not be by Kaz's hand and was therefore unacceptable. In fact, many of Kaz's mistakes can be led back to his maddened need for vengeance.
Similarly, his gloves make people fear him. They add to his mystery and aura of danger. However, their true purpose is to protect him from touch. His gloves are a symbol of his debilitating weakness, his inability to handle any sort of skin on skin contact. What he uses to scare people is used to hide his own terror.
The alias Brekker is similar. He associates the name Rietveld with weakness and naivety, and it is tied to the memory of falling for a con. He replaces it, and his personality, with something harsh and ominous. He is no longer Kaz Rietveld - naive farm boy; he is Kaz Brekker - hardened criminal and conman. However his need to change his name stemmed from his failure - the need to conceal himself so that Pekka wouldn't see him coming.
The cane, on the other hand, is a weakness he makes into a strength rather than the opposite way around. It's the only time people saw the source of the persona before the persona itself. The anger, the gloves, the name - this all came before Kaz reached notoriety and are internal. The cane came after. It was the first time Kaz had to spin his persona with people watching. Because it's always the weakness that comes first. However, the cane is harder to hide. That's possibly why the associations he creates with the cane are more actively violent - a weapon. Not a warning like the gloves or the anger or the name. Instead, it is an active tool he uses to create harm. When people are watching, he immediately turns to violence.