This isn't the first controversy to hit the Tokyo Olympics.
In July, Prime Minister Shinzo Abe announced the country would scrap plans for an expensive new stadium for the 2020 showpiece.
Abe said the design was dropped due to skyrocketing costs, which rose to 250 billion yen ($2.02 billion) at one point.
The design, by Iraqi-British architect Zaha Hadid, had drawn scathing criticism for its apparent likeness to a bike helmet or a toilet seat.
Baseball, bowling and surfing among contenders for Tokyo Olympics
Sano said he made the decision to scrap the Tokyo logo in part to "protect his family and staff from criticism and bashings," adding "I have reached my limit where I can no longer endure as a human being.