Clinical reports and scientific health information sources are usually written for medical experts preventing patients from understanding the main messages of these texts. Making them comprehensible for patients is important to enable patients to make informed health decisions. Metrics are required to assess readability and to evaluate text simplification methods. However, research has mainly focused on English medical texts. We collected a set of 18 statistical, part-of-speech-based, syntactic, semantic and fluency metrics from related studies and evaluate their suitability to measure readability of German medical texts. We perform multiple t-tests on technical abstracts from English and German scientific articles and related simplified summaries, respectively. While semantic and fluency metrics can be successfully transferred to German medical texts, multiple statistical, part-of-speech-based, and syntactic metrics behave differently when they are applied to German medical texts requiring careful interpretation.