Stone Town is known for its narrow alleyways, large carved doors and covered balconies. The doors, large wooden carved affairs with or without brass studs, are a part of the Swahili culture influenced by Arab and especially Indian motifs. The large brass studs became decoration after first having served as spike covers; the spikes having been protection from elephant raids during wars in India. Doors with rounded tops, or lintels, reflect Indian influence while doors with flat lintels demonstrate a version popular with Omanis in Zanzibar. Many doors have Koranic inscriptions and some of the older doors found in town are much less ornate than the later ones. Different carvings to look for are chains around the edge meant to bring security, Lotus and rosettes in the center meant to represent prosperity, and fish at the bottom representing fertility.

On the waterfront, near the Old Dispensary, is an old tree known locally as the Big Tree. Some locals believe that Sultan Khalifa planted it in 1911 but others believe it was planted in 1944 as a bicentennial of Al Busaid. The Big Tree is quite visible from the harbor and is seen in many old photographs. The shaded area underneath it is currently used as a workshop for men building boats.

Only 226 or about 13 per cent of Stone Town's buildings are considered to be in good condition – the remaining structures are either deteriorating or in ruins.