This paper discusses an informal methodology for evaluating Machine Translation software documentation with reference to a case study, in which a number of currently available MT packages are evaluated. Different types of documentation style are discussed, as well as different user profiles. It is found that documentation is often inadequate in identifying the level of linguistic background and knowledge necessary to use translation software, and in explaining technical (linguistic) terms needed to use the software effectively. In particular, the level of knowledge and training needed to use the software is often incompatible with the user profile implied by the documentation. Also, guidance on how to perform more complex tasks, which may be especially idiosyncratic, is often inadequate or missing altogether.