Archaeological excavations and research have revealed the remains of a castle built in the year 1192. It was destroyed in 1430 by the Hussites. The next phase was the construction of a residence built in a mixture of the late-Gothic and Renaissance styles. This was heavily damaged during the Thirty Years' War (1618 - 1648).
Castle Planitz in its present-day state dates back to 1700 - 1720. It was constructed at the behest of the von Arnim family as a four-wing palace in the Baroque style. After the Planitz line of the von Arnim died out, the castle was bought by the Planitzer Sparkasse (savings bank). From 1935, it was rented to Planitz council as their town hall. It later served as an army hospital, as the Soviet military command centre, as a retirement home for miners, as a barracks and as a police academy.
In 1993, it was acquired by the city on behalf of the Clara Wieck Grammar School. This is one of only three educational facilities in the Free State of Saxony which specialise in music.