It extends in a broad valley surrounded by the hills of Podkrkonošská pahorkatina. The first record of Semily dates back to 1352. As it refers to a church, it may be assumed that Semily had existed long before.

For centuries Semily used to be a residence of the lords of the manor. The manor was owned by the Smiřický family for a long time, then by Albrecht of Waldstein and later it was held by foreign noble families. The remains of the Semily manor were bought by the town of Semily in 1890.

Thanks to its location, Semily is a good starting point for excursions to Český ráj, the southern part of the Jizerské hory and Podkrkonoší.