The Castell de Vilafamés stands in the highest part of the town, in the region of La Plana Alta. Of Arab origin (Beni-Hamez), it was conquered in 1233 and built between the twelfth and fourteenth centuries, being remodeled in the nineteenth century. It offers panoramic views of the village and the surrounding mountains. In a small rocky concavity at the top of the castle there are cave paintings.