The use of domain-specific modeling languages (DSMLs) results in higher productivity during the development process. This is accomplished by raising the level of abstraction during design and focusing on domain concepts rather than low-level implementation details. Unlike other development paradigms, little work has been done in determining and measuring the claimed benefits of using DSMLs. In this paper, we propose a new approach to determine the effort involved in creating and using DSML models to develop applications and to manage the behavior of applications at runtime. The approach involves a classification of the effort involved, and definition of relevant metrics to measure the effort for each category. A case study is presented that shows how we applied the proposed metrics during the development and execution of an application using three different DSMLs. Copyright 2010 ACM.