Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

Is SMS APPropriate?

Comparative Properties of SMS and Apps for Repeated Measures Data Collection

Published Online:December 29, 2016https://doi.org/10.1027/1015-5759/a000376

Abstract

Abstract. The ubiquity of mobile telephones worldwide offers a unique opportunity for bidirectional communication between researchers and participants. There are two ways mobile phones could be used to collect self-report data: via Short Message Service (SMS) or app (mobile telephone software applications). This study examined the comparative data quality offered by SMS and app, when mobile phone type, self-report instrument, and sampling schedule are controlled. One hundred ten undergraduate students used their own iPhones to complete the same repeated measures instrument on 20 occasions, responding either by SMS or by app. There were no differences between SMS and app respondents in terms of response rates or response delay. However, data from those responding via SMS was significantly less complete than from app respondents. App respondents rated their respondent experience as more convenient than SMS respondents. Though findings are only generalizable to an undergraduate sample, this suggests that researchers should consider using apps rather than SMS for repeated measures self-report data collection.

References

  • ACMA. (2013). ACMA Communications report 2012–2013. Australia: The Australian Media Communications Authority. First citation in articleGoogle Scholar

  • Akamatsu, C. T., Mayer, C. & Farrelly, S. (2006). An investigation of two-way text messaging use with deaf students at the secondary level. Journal of Deaf Studies and Deaf Education, 11, 120–131. doi: 10.1093/deafed/enj013 First citation in articleCrossref, Google Scholar

  • Anhoj, J. & Moldrup, C. (2009). Feasibility of collecting diary data from asthma patients through mobile phones and SMS (short message service): Response rate analysis and focus group evaluation from a pilot study. Journal of Medical Internet Research, 6, e42. doi: 10.2196/jmir.6.4.e42 First citation in articleCrossref, Google Scholar

  • Asiimwe, C., Gelvin, D., Lee, E., Ben Amor, Y., Quinto, E., Katureebe, C., … Berg, M. (2011). Use of an innovative, affordable, and open-source short message service-based tool to monitor malaria in remote areas of Uganda. The American Journal of Tropical Medicine and Hygiene, 85, 26–33. doi: 10.4269/ajtmh.2011.10-0528 First citation in articleCrossref, Google Scholar

  • Baruch, Y. & Holtom, B. C. (2008). Survey response rate levels and trends in organizational research. Human Relations, 61, 1139–1160. doi: 10.1177/0018726708094863 First citation in articleCrossref, Google Scholar

  • Broderick, C. R., Herbert, R. D., Latimer, J., Mathieu, E., van Doorn, N. & Curtin, J. A. (2012). Feasibility of short message service to document bleeding episodes in children with haemophilia. Haemophilia: The Official Journal of the World Federation of Hemophilia, 18, 906–910. doi: 10.1111/j.1365-2516.2012.02869.x First citation in articleCrossref, Google Scholar

  • Charlton, T., Panting, C. & Hannan, A. (2002). Mobile telephone ownership and usage among 10- and 11-year-olds. Emotional and Behavioural Difficulties, 7, 37–41. First citation in articleGoogle Scholar

  • Chib, A., Wilkin, H., Ling, L. X., Hoefman, B. & Van Biejma, H. (2012). You have an important message! Evaluating the effectiveness of a text message HIV/AIDS campaign in Northwest Uganda. Journal of Health Communication, 17(Suppl 1(April 2014)), 146–157. doi: 10.1080/10810730.2011.649104 First citation in articleCrossref, Google Scholar

  • Conner, T. S. & Reid, K. A. (2012). Effects of intensive mobile happiness reporting in daily life. Social Psychological and Personality Science, 3, 315–323. First citation in articleCrossref, Google Scholar

  • Cook, C., Heath, F. & Thompson, R. L. (2000). A meta-analysis of response rates in Web- or Internet-based surveys. Educational and Psychological Measurement, 60, 821–836. doi: 10.1177/00131640021970934 First citation in articleCrossref, Google Scholar

  • Déglise, C., Suggs, L. S. & Odermatt, P. (2012). Short message service (SMS) applications for disease prevention in developing countries. Journal of Medical Internet Research, 14, e3. doi: 10.2196/jmir.1823 First citation in articleCrossref, Google Scholar

  • De Lepper, A. M., Eijkemans, M. J. C., Van Beijma, H., Loggers, J. W. & Tuijn, C. J. (2013). Response patterns to interactive SMS health education quizzes at two sites in Uganda: A cohort study. Tropical Medicine and International Health, 18, 516–521. doi: 10.1111/tmi.12059 First citation in articleCrossref, Google Scholar

  • Deloitte Global Media Consumer Survey, Developed Countries. (2013). United Kingdom: Deloitte Touche Tohmatsu Ltd.. First citation in articleGoogle Scholar

  • Devitt, K. & Roker, D. (2009). The role of mobile phones in family communication. Children & Society, 23, 189–202. doi: 10.1111/j.1099-0860.2008.00166.x First citation in articleCrossref, Google Scholar

  • Dillman, D., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J. & Messer, B. L. (2009a). Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet. Social Science Research, 38, 1–18. doi: 10.1016/j.ssresearch.2008.03.007 First citation in articleCrossref, Google Scholar

  • Dillman, D. A., Smyth, J. D. & Christian, L. M. (2009b). Internet, Mail, and Mixed-Mode Surveys (3rd ed.). Hoboken, NJ: Wiley. First citation in articleGoogle Scholar

  • Donaldson, E. L., Fallows, S. & Morris, M. (2014). A text message based weight management intervention for overweight adults. Journal of Human Nutrition and Dietetics: The Official Journal of the British Dietetic Association, 27(Suppl 2), 90–97. doi: 10.1111/jhn.12096 First citation in articleCrossref, Google Scholar

  • Fernandez, K. C., Johnson, M. R. & Rodebaugh, T. L. (2013). TelEMA: A low-cost and user-friendly telephone assessment platform. Behavior Research Methods, 45, 1279–1291. doi: 10.3758/s13428-012-0287-9 First citation in articleCrossref, Google Scholar

  • Flick, S. N. (1988). Managing attrition in clinical research. Clinical Psychology Review, 8, 499–515. doi: 10.1016/0272-7358(88)90076-1 First citation in articleCrossref, Google Scholar

  • Fox, R., Crask, M. & Kim, J. (1988). Mail survey response rate a meta-analysis of selected techniques for inducing response. Public Opinion Quarterly, 52, 467–491. First citation in articleCrossref, Google Scholar

  • Fukuoka, Y. & Kamitani, E. (2011). New insights into compliance with a mobile phone diary and pedometer use in sedentary women. Journal of Physical Activity and Health, 8, 398–403. First citation in articleCrossref, Google Scholar

  • Haller, D., Sanci, L., Sawyer, S., Coffey, C. & Patton, G. (2006). R U OK 2 TXT 4 RESEARCH? – Feasibility of text message communication in primary care research. Australian Family Physician, 35, 175–176. First citation in articleGoogle Scholar

  • Hofmann, W. & Patel, P. V. (2014). SurveySignal: A convenient solution for experience sampling research using participants’ own smartphones. Social Science Computer Review, 33, 235–253. doi: 10.1177/0894439314525117 First citation in articleCrossref, Google Scholar

  • Keijzers, J., Ouden, E. D. & Lu, Y. (2008). Usability benchmark study of commercially available smart phones: Cell phone type platform, PDA type platform and PC type platform. Proceedings of the 10th International Conference on Human Computer Interaction with Mobile Devices and Services (pp. 265–272). ACM. Retrieved from http://dl.acm.org/citation.cfm?id=1409269 First citation in articleCrossref, Google Scholar

  • Kojo, I., Heiskala, M. & Virtanen, J. (2014). Customer journey mapping of an experience-centric service by mobile self-reporting: Testing the Qualiwall Tool. In A. MarcusEd., Design, user experience, and usability. Theories, methods, and tools for designing the user experience (pp. 261–272). Bern, Switzerland: Springer International. Retrieved from http://link.springer.com/chapter/10.1007/978-3-319-07668-3_26 First citation in articleGoogle Scholar

  • Lehman, B. J. (2011). Getting started: Launching a study in daily life. In M. R. MehlT. S. ConnerEds., Handbook of research methods for studying daily life (pp. 89–107). New York, NY: The Guilford Press. First citation in articleGoogle Scholar

  • Lim, M. S. C., Sacks-Davis, R., Aitken, C. K., Hocking, J. S. & Hellard, M. E. (2010). Randomised controlled trial of paper, online and SMS diaries for collecting sexual behavior information from young people. Journal of Epidemiology and Community Health, 64, 885–889. doi: 10.1136/jech.2008.085316 First citation in articleCrossref, Google Scholar

  • Ling, R. (2010). Texting as a life phase medium. Journal of Computer-Mediated Communication, 15, 277–292. doi: 10.1111/j.1083-6101.2010.01520.x First citation in articleCrossref, Google Scholar

  • Lobet-Maris, C. & Henin, L. (2002). Talking without communicating or communicating without talking: From the GSM to the SMS. In Estudios de Juventud, 57, 101–114). First citation in articleGoogle Scholar

  • Mackay, M. M. & Weidlich, O. (2009). Austrailan Mobile Phone lifestyle index. Specialist. Australian Interactive Media Industry Association Mobile Industry Group. Retrieved from http://www.aimia.com.au/enews/mobile/090929AIMIA_Report_FINAL.pdf First citation in articleGoogle Scholar

  • Mallenius, S., Rossi, M. & Tuunainen, V. (2007). Factors affecting the adoption and use of mobile devices and services by elderly people – Results from a pilot study. 6th Annual Global Mobility Roundtable, 31. Retrieved fromhttp://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.130.2463&rep=rep1&type=pdf First citation in articleGoogle Scholar

  • Mante, E. A. & Piris, D. (2010). SMS use by young people in the Netherlands. Revista de Estudios de Juventud, 52, 47–58. First citation in articleGoogle Scholar

  • Marshall, A., Medvedev, O. & Antonov, A. (2008). Use of a smartphone for improved self-management of pulmonary rehabilitation. International Journal of Telemedicine and Applications, , 1–5. doi: 10.1155/2008/753064 First citation in articleCrossref, Google Scholar

  • Matthews, M., Doherty, G., Sharry, J. & Fitzpatrick, C. (2008). Mobile phone mood charting for adolescents. British Journal of Guidance & Counselling, 36, 113–129. doi: 10.1080/03069880801926400 First citation in articleCrossref, Google Scholar

  • Miller, G. (2012). The smartphone psychology manifesto. Perspectives on Psychological Science, 7, 221–237. doi: 10.1177/1745691612441215 First citation in articleCrossref, Google Scholar

  • Mogensen, A. (1963). Item-skipping and right and wrong solutions in a preliminary version of a multiple-choice vocabulary test. Acta Psychologica, 21, 49–54. First citation in articleCrossref, Google Scholar

  • Montag, C., Błaszkiewicz, K., Lachmann, B., Andone, I., Sariyska, R., Trendafilov, B., … Markowetz, A. (2014). Correlating personality and actual phone usage: Evidence from psychoinformatics. Journal of Individual Differences, 35, 158–165. First citation in articleLink, Google Scholar

  • Morris, M. E., Kathawala, Q., Leen, T. K., Gorenstein, E. E., Guilak, F., Labhard, M. & Deleeuw, W. (2010). Mobile therapy: Case study evaluations of a cell phone application for emotional self-awareness. Journal of Medical Internet Research, 12, e10. doi: 10.2196/jmir.1371 First citation in articleCrossref, Google Scholar

  • Naughton, F., Jamison, J. & Sutton, S. (2013). Attitudes towards SMS text message smoking cessation support: A qualitative study of pregnant smokers. Health Education Research, 28, 911–922. doi: 10.1093/her/cyt057 First citation in articleCrossref, Google Scholar

  • Pain, R., Grundy, S. U. E., Gill, S., Towner, E., Sparks, G. & Hughes, K. (2005). “So long as i take my mobile”: Mobile phones, urban life and geographies of young people‘s safety. International Journal of Urban and Regional Research, 29, 814–830. First citation in articleCrossref, Google Scholar

  • Ranney, M. L., Choo, E. K., Cunningham, R. M., Spirito, A., Thorsen, M., Mello, M. J. & Morrow, K. (2014). Acceptability, language, and structure of text message-based behavioral interventions for high-risk adolescent females: A qualitative study. Journal of Adolescent Health, 55, 1–8. doi: 10.1016/j.jadohealth.2013.12.017 First citation in articleCrossref, Google Scholar

  • Raphael, K. (1987). Recall bias: A proposal for assessment and control. International Journal of Epidemiology, 16, 167–170. First citation in articleCrossref, Google Scholar

  • Reimers, S. & Stewart, N. (2009). Using SMS text messaging for teaching and data collection in the behavioral sciences. Behavior Research Methods, 41, 675–681. doi: 10.3758/BRM.41.3.675 First citation in articleCrossref, Google Scholar

  • Ribeiro, A. & da Silva, A. R. (2012). Survey on cross-platforms and languages for mobile apps. 2012 Eighth International Conference on the Quality of Information and Communications Technology. Washington, DC: IEEE, 255–260. doi: 10.1109/QUATIC.2012.56 First citation in articleCrossref, Google Scholar

  • Richardson, C. & Johnson, J. (2009). The influence of web- versus paper-based formats on the assessment of tobacco dependence: Evaluating the measurement invariance of the Dimensions of Tobacco Dependence Scale. Substance Abuse: Research and Treatment, 3, 1–14. First citation in articleCrossref, Google Scholar

  • Rosser, B. A. & Eccleston, C. (2011). Smartphone applications for pain management. Journal of Telemedicine and Telecare, 17, 308–312. doi: 10.1258/jtt.2011.101102 First citation in articleCrossref, Google Scholar

  • Sax, L., Gilmartin, S. & Bryant, A. (2003). Assessing response rates and nonresponse bias in web and paper surveys. Research in Higher Education, 44, 409–432. doi: 10.1023/A:1024232915870 First citation in articleCrossref, Google Scholar

  • Sharp, L. M. & Frankel, J. (1983). Respondent burden: A test of some common assumptions. Public Opinion Quarterly, 47, 36–53. First citation in articleCrossref, Google Scholar

  • The World Bank. (2012). 2012 Information and Communications for Development: Maximizing Mobile, Washington, DC: World Bank. First citation in articleCrossref, Google Scholar

  • Tsai, C. C., Lee, G., Raab, F., Norman, G. J., Sohn, T., Griswold, W. G. & Patrick, K. (2007). Usability and feasibility of PmEB: A mobile phone application for monitoring real time caloric balance. Mobile Networks and Applications, 12, 173–184. doi: 10.1007/s11036-007-0014-4 First citation in articleCrossref, Google Scholar

  • Van Buuren, S. (2010). Item imputation without specifying scale structure. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 6, 31–36. doi: 10.1027/1614-2241/a000004 First citation in articleLink, Google Scholar

  • Van de Vijver, F. & Harsveldt, M. (1994). The incomplete equivalence of the paper-and-pencil and computerized versions of the General Aptitude Test Battery. Journal of Applied Psychology, 79, 852–859. First citation in articleCrossref, Google Scholar

  • Walsh, E. I. & Brinker, J. K. (2012). Evaluation of a Short Message Service diary methodology in a nonclinical, naturalistic setting. Cyberpsychology, Behavior and Social Networking, 15, 615–618. doi: 10.1089/cyber.2012.0189 First citation in articleCrossref, Google Scholar

  • Yarkoni, T. (2012). Psychoinformatics new horizons at the interface of the psychological and computing sciences. Current Directions in Psychological Science, 21, 391–397. First citation in articleCrossref, Google Scholar