Guidelines on Conducting Online Experiments with Repeated Measures and Using Smartphone Applications

[Panduan Melakukan Eksperimen Daring dengan Pengukuran Multipel serta Menggunakan Aplikasi Ponsel]

  • Intan Permatasari Universitas Indonesia
  • Harry Susianto Universitas Indonesia
Abstract Views: 305 PDF - Full Text Downloads: 169
Keywords: online experiment, eksperimen daring, repeated measures, pengukuran multipel, smartphone application, aplikasi ponsel

Abstract

In Indonesia, online research is typically conducted using survey design. Data collection is conducted by distributing links to questionnaires through social media. Compared to surveys, online experimental research, especially ones that utilize mobile applications and require repeated measures, are very rarely implemented. This is thought to be caused by the lack of experience among researchers of psychology in creating and managing online research platforms or smartphone applications. This article aims to guide researchers on conducting online experiments with repeated measures, integrated with additional smartphone applications that are available for free.

Di Indonesia penelitian secara daring umumnya dilakukan menggunakan desain survei. Pengumpulan datanya dilakukan dengan cara menyebarkan tautan kuesioner melalui media sosial. Dibandingkan survei, penelitian eksperimen daring, terlebih lagi yang menggunakan aplikasi ponsel dan membutuhkan pengukuran multipel sangat jarang dilakukan. Penyebabnya diduga karena peneliti psikologi umumnya belum terbiasa membuat serta mengelola platform penelitian daring atau aplikasi ponsel. Artikel ini bertujuan memberikan panduan melakukan eksperimen daring dengan pengukuran multipel (repeated measures) yang mengintegrasikan tambahan aplikasi ponsel yang bisa dilakukan secara gratis.

Downloads

Download data is not yet available.

References

Abshire, M., Dinglas, V. D., Cajita, M. I. A., Eakin, M. N., Needham, D. M., & Himmelfarb, C. D. (2017). Participant retention practices in longitudinal clinical research studies with high retention rates. BMC Medical Research Methodology, 17:30, 1-10. https://doi.org/10.1186/s12874-017-0310-z

Allen, S., Graupera, V., & Lundrigan, L. (2010). The smartphone is the new PC. In Pro smartphone cross-platform development (pp. 1-14). Apress. https://doi.org/10.1007/978-1-4302-2869-1_1

Amazon Mechanical Turk. (2005). Features. Amazon Mechanical Turk (MTurk). https://www.mturk.com/product-details

Amazon Mechanical Turk. (2017, March 14). MTurk is now available for requesters in 10 more countries. Amazon Mechanical Turk. https://blog.mturk.com/mturk-is-now-available-for-requesters-in-10-more-countries-a4e1fd173f0b

American Psychological Association [APA]. (2020). Publication manual of the American Psychological Association (7th ed.). https://apastyle.apa.org/products/publication-manual-7th-edition

American Psychological Association [APA]. (n.d.). Apparatus. APA Dictionary of Psychology. Retrieved August 17, 2021 from https://dictionary.apa.org/apparatus

Anjum, A., & Ilyas, M. U. (2013). Activity recognition using smartphone sensors. Proceedings of 2013 IEEE 10th Consumer Communications and Networking Conference (CCNC), 914-919. https://doi.org/10.1109/CCNC.2013.6488584

Anwyl-Irvine, A., Dalmaijer, E. S., Hodges, N., & Evershed, J. K. (2021). Realistic precision and accuracy of online experiment platforms, web browsers, and devices. Behavior Research Methods, 53(4), 1407-1425. https://doi.org/10.3758/s13428-020-01501-5

Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board Task Force report. American Psychologist, 73(1), 3-25. https://doi.org/10.1037/amp0000191

Atzmüller, C., & Steiner, P. M. (2010). Experimental vignette studies in survey research. Methodology, 6(3), 128-138. https://doi.org/10.1027/1614-2241/a000014

Aziz, B. G. A., & Lei, S. (2016). Research on the smartphone fast growing impact on mobile gaming industry in China. Journal of Management, Marketing and Logistics, 3(4), 368-368. https://doi.org/10.17261/pressacademia.2016.341

Becker, R. A., Chambers, J. M., & Wilks, A. R. (1988). The new S language: A programming environment for data analysis and graphics (1st ed.). Wadsworth & Brooks/Cole Advanced Books & Software. https://www.routledge.com/The-New-S-Language/Becker/p/book/9781315895888

Boehmke, B. C. (2016). Data wrangling with R. Springer International Publishing. https://doi.org/10.1007/978-3-319-45599-0

Butler, J., & Kern, M. L. (2016). The PERMA-profiler: A brief multidimensional measure of flourishing. International Journal of Wellbeing, 6(3), 1-48. https://doi.org/10.5502/ijw.v6i3.526

Cantor, D., O’Hare, B. C., & O’Connor, K. S. (2008). The use of monetary incentives to reduce nonresponse in random digit dial telephone surveys. In J. M. Lepkowski, C. Tucker, J. M. Brick, E. D. de Leeuw, L. Japec, P. J. Lavrakas, M. W. Link, & R. L. Sangster (Eds.), Advances in telephone survey methodology (pp. 471-498). Wiley. https://doi.org/10.1002/9780470173404.ch22

Capon, H., Hall, W., Fry, C., & Carter, A. (2016). Realising the technological promise of smartphones in addiction research and treatment: An ethical review. International Journal of Drug Policy, 36, 47-57. https://doi.org/10.1016/j.drugpo.2016.05.013

Chan, D. (1998). The conceptualization and analysis of change over time: An integrative approach incorporating Longitudinal Mean and Covariance Structures Analysis (LMACS) and Multiple Indicator Latent Growth Modeling (MLGM). Organizational Research Methods, 1(4), 421-483. https://doi.org/10.1177/109442819814004

Clough, A., Wagman, J., Rollins, C., Barnes, J., Connor-Smith, J., Holditch-Niolon, P., McDowell, S., Martinez-Bell, E., Bloom, T., Baker, C., & Glass, N. (2010). The SHARE project: Maximizing participant retention in a longitudinal study with victims of intimate partner violence. Field Methods, 23(1), 86-101. https://doi.org/10.1177/1525822X10384446

Cook, D. A., Wittich, C. M., Daniels, W. L., West, C. P., Harris, A. M., & Beebe, T. J. (2016). Incentive and reminder strategies to improve response rate for Internet-based physician surveys: A randomized experiment. Journal of Medical Internet Research (JMIR), 18(9), 1-11. https://doi.org/10.2196/jmir.6318

Cotter, R. B., Burke, J. D., Loeber, R., & Navratil, J. L. (2002). Innovative retention methods in longitudinal research: A case study of the developmental trends study. Journal of Child and Family Studies, 11(4), 485-498. https://doi.org/10.1023/A:1020939626243

den Eynden, V. V, Corti, L., Woollard, M., Bishop, L., & Horton, L. (2011). Managing and sharing data: Best practice for researchers (3rd ed.). UK Data Archive - University of Essex. http://www.data-archive.ac.uk/media/2894/managingsharing.pdf

Duarte, F. (2019, September 9). Berapa banyak waktu yang dihabiskan rakyat Indonesia di media sosial? [How much time do Indonesians spend on social media?]. BBC News Indonesia. https://www.bbc.com/indonesia/majalah-49630216

Elliott, W. B., Hodge, F. D., & Sedor, L. M. (2012). Using online video to announce a restatement: Influences on investment decisions and the mediating role of trust. The Accounting Review, 87(2), 513-535. https://doi.org/10.2308/accr-10202

Farrington, D. P. (1991). Longitudinal research strategies: Advantages, problems, and prospects. Journal of the American Academy of Child & Adolescent Psychiatry, 30(3), 369-374. https:doi.org/10.1097/00004583-199105000-00003

Google. (n.d.a). Show questions based on answers. Google. Retrieved March 14, 2021 from https://support.google.com/docs/answer/141062?hl=en

Google. (n.d.b). Cara menggunakan Google Formulir [How to use Google Forms]. Google. Retrieved March 14, 2021 from https://support.google.com/docs/answer/6281888?hl=id&co=GENIE.Platform%3DDesktop

Google. (n.d.c). Google Formulir [Google Forms]. Google. Retrieved March 14, 2021 from https://www.google.com/intl/id_id/forms/about/

Google. (n.d.d). Choose where to save form responses. Google. Retrieved March 14, 2021 from https://support.google.com/docs/answer/2917686?hl=en#zippy=%2Cchoose-where-to-store-responses%2Cdelete-all-responses-from-a-form

Gustavson, K., von Soest, T., Karevold, E., & Røysamb, E. (2012). Attrition and generalizability in longitudinal studies: Findings from a 15-year population-based study and a Monte Carlo simulation study. BMC Public Health, 12:918, 1-11. https://doi.org/10.1186/1471-2458-12-918

Harari, G. M., Lane, N. D., Wang, R., Crosier, B. S., Campbell, A. T., & Gosling, S. D. (2016). Using smartphones to collect behavioral data in psychological science: Opportunities, practical considerations, and challenges. Perspectives on Psychological Science, 11(6), 838-854. https://doi.org/10.1177/1745691616650285

Hartman, S. J., Nelson, S. H., Cadmus-Bertram, L. A., Patterson, R. E., Parker, B. A., & Pierce, J. P. (2016). Technology- and phone-based weight loss intervention: Pilot RCT in women at elevated breast cancer risk. American Journal of Preventive Medicine, 51(5), 714-721. https://doi.org/10.1016/j.amepre.2016.06.024

Hiskey, S., & Troop, N. A. (2002). Online longitudinal survey research: Viability and participation. Social Science Computer Review Hiskey, 20(3), 250-259. https://doi.org/10.1177/089443930202000303

Hofer, S. M., & Hoffman, L. (2007). Statistical analysis with incomplete data: A developmental perspective. In T. D. Little, J. A. Bovaird, & N. A. Card (Eds.), Modeling contextual effects in longitudinal studies (pp. 13-32). Lawrence Erlbaum Associates Publishers. https://psycnet.apa.org/record/2007-07587-002

Hooley, T., Wellens, J., & Marriott, J. (2012). What is online research? Using the internet for social science research. Bloomsbury Publishing. https://www.bloomsbury.com/uk/what-is-online-research-9781849665247/

Information Commissioner’s Office [ICO]. (n.d.). What are identifiers and related factors?. Information Commissioner’s Office. Retrieved March 14, 2021 from https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection-regulation-gdpr/what-is-personal-data/what-are-identifiers-and-related-factors/

Janke, R. (2014). Effects of mentioning the incentive prize in the email subject line on survey response. Evidence Based Library and Information Practice, 9(1), 4-13. https://doi.org/10.18438/b8vw27

Jaya, E. S., Ascone, L., & Lincoln, T. M. (2018). A longitudinal mediation analysis of the effect of negative-self-schemas on positive symptoms via negative affect. Psychological Medicine, 48(8), 1299-1307. https://doi.org/10.1017/S003329171700277X

Joint Information Systems Committee [JISC]. (2018). Research data management toolkit. Joint Information Systems Committee. https://www.jisc.ac.uk/guides/rdm-toolkit

Kirono, D. C. G., & Supangat, A. (2017, December 21). Media sosial, metode riset baru ala Citizen Science. [Social media, Citizen Science's new research method.]. Mongabay: Situs Berita Lingkungan. https://www.mongabay.co.id/2017/12/21/media-sosial-metode-riset-baru-ala-citizen-science/

Kramer, I., Simons, C. J. P., Hartmann, J. A., Menne-Lothmann, C., Viechtbauer, W., Peeters, F., Schruers, K., van Bemmel, A. L., Myin-Germeys, I., & Delespaul, P. (2014). A therapeutic application of the experience sampling method in the treatment of depression: A randomized controlled trial. World Psychiatry, 13(1), 68-77. https://doi.org/10.1002/wps.20090

Kraut, R., Olson, J., Banaji, M., Bruckman, A., Cohen, J., & Couper, M. (2004). Psychological research online: Report of Board of Scientific Affairs’ Advisory Group on the conduct of research on the internet. American Psychologist, 59(2), 105-117. https://doi.org/10.1037/0003-066X.59.2.105

Kreuter, F., Haas, G. -C., Keusch, F., Bähr, S., & Trappmann, M. (2020). Collecting survey and smartphone sensor data with an app: Opportunities and challenges around privacy and informed consent. Social Science Computer Review, 38(5), 533-549. https://doi.org/10.1177/0894439318816389

Lane, N. D., Miluzzo, E., Lu, H., Peebles, D., Choudhury, T., & Campbell, A. T. (2010). A survey of mobile phone sensing. IEEE Communications Magazine, 40(9), 140-150. https://doi.org/10.1109/MCOM.2010.5560598

Lane, N. D., Mohammod, M., Lin, M., Yang, X., Lu, H., Ali, S., Doryab, A., Berke, E., Choudhury, T., & Campbell, A. (2011, May 23-26). BeWell: A smartphone application to monitor, model and promote wellbeing. Proceedings of 5th International ICST Conference on Pervasive Computing Technologies for Healthcare, Pervasive Health, Dublin, Republic of Ireland. European Union Digital Library. https://doi.org/10.4108/icst.pervasivehealth.2011.246161

Lin, C. -C., Chang, C. -C., Liang, D., & Yang, C. -H. (2012). A new non-intrusive authentication method based on the orientation sensor for smartphone users. Proceedings of the 2012 IEEE 6th International Conference on Software Security and Reliability, 245–252. https://doi.org/10.1109/SERE.2012.37

Martin, A. A., Grünenfelder-Steiger, A. E., & Allemand, M. (2020). Longitudinal research. In B. J. Carducci, & C. S. Nave (Eds.), The Wiley encyclopedia of personality and individual differences: Measurement and assessment (pp. 25-30). Wiley. https://doi.org/10.1002/9781118970843.ch74

Martinez-Martin, N., & Kreitmair, K. (2018). Ethical issues for direct-to-consumer digital psychotherapy apps: Addressing accountability, data protection, and consent. JMIR Mental Health, 5(2):e32, 1-7. https://doi.org/10.2196/mental.9423

McKinney, W. (2017). Python for data analysis: Data wrangling with Pandas, Numpy, and IPython (2nd ed.). O'Reilly Media. https://www.oreilly.com/library/view/python-for-data/9781491957653/

Miller, G. (2012). The smartphone psychology manifesto. Perspectives on Psychological Science, 7(3), 221-237. https://doi.org/10.1177/1745691612441215

Mourby, M., Gowans, H., Aidinlis, S., Smith, H., & Kaye, J. (2019). Governance of academic research data under the GDPR-lessons from the UK. International Data Privacy Law, 9(3), 192-206. https://doi.org/10.1093/IDPL/IPZ010

Musch, J., & Reips, U. D. (2000). A brief history of web experimenting. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 61-87). Elsevier. https://www.elsevier.com/books/psychological-experiments-on-the-internet/birnbaum/978-0-12-099980-4

Mychasiuk, R., & Benzies, K. (2012). Facebook: An effective tool for participant retention in longitudinal research. Child: Care, Health and Development, 38(5), 753-756. https://doi.org/10.1111/J.1365-2214.2011.01326.X

Natalya, L. (2018). Validation of Academic Motivation Scale: Short Indonesian language version. ANIMA Indonesian Psychological Journal, 34(1), 43-53. https://doi.org/10.24123/aipj.v34i1.2025

OpenAire. (2021). Guide for researcher: How to deal with sensitive data. OpenAire. Retrieved August 3, 2021 from https://www.openaire.eu/sensitive-data-guide

Pertiwi, W. K. (2019, January 18). WhatsApp kini lebih populer dari Facebook [WhatsApp is now more popular than Facebook]. Kompas. https://tekno.kompas.com/read/2019/01/18/09260907/whatsapp-kini-lebih-populer-dari-facebook

Pinder, C., Vermeulen, J., & Cowan, B. R. (2018). Subliminal semantic number processing on smartphones. Proceedings of 20th International Conference on Human-Computer Interaction with Mobile Devices and Services, Spain, e24, 1-13. https://doi.org/10.1145/3229434.3229451

Pinder, C., Vermeulen, J., Beale, R., & Hendley, R. (2015, August 25-28). Subliminal priming of nonconscious goals on smartphones [Poster session]. Proceedings of 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunc, Coppenhagen, Denmark, 825-830. https://doi.org/10.1145/2786567.2793707

Pinder, C., Vermeulen, J., Cowan, B. R., Beale, R., & Hendley, R. J. (2017). Exploring the feasibility of subliminal priming on smartphones. Proceedings of 19th International Conference on Human-Computer Interaction with Mobile Devices and Services, Austria, e21, 1-15. https://doi.org/10.1145/3098279.3098531

Piwek, L., Ellis, D. A., & Andrews, S. (2016). Can programming frameworks bring smartphones into the mainstream of psychological science? Frontiers in Psychology, 7:1252. https://doi.org/10.3389/fpsyg.2016.01252

Poon, K. -T., Teng, F., Wong, W. -Y., & Chen, Z. (2016). When nature heals: Nature exposure moderates the relationship between ostracism and aggression. Journal of Environmental Psychology, 48, 159-168. https://doi.org/10.1016/j.jenvp.2016.10.002

Pouchard, L. (2015). Revisiting the data lifecycle with big data curation. International Journal of Digital Curation, 10(2), 176-192. https://doi.org/10.2218/ijdc.v10i2.342

Purves, K. L., Constantinou, E., McGregor, T., Lester, K. J., Barry, T. J., Treanor, M., Sun, M., Margraf, J., Craske, M. G., Breen, G., & Eley, T. C. (2019). Validating the use of a smartphone app for remote administration of a fear conditioning paradigm. Behaviour Research and Therapy, 123:103475. https://doi.org/10.1016/j.brat.2019.103475

R Core Team. (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved March 16, 2021. Available online from https://www.r-project.org/

Rancangan Undang-Undang Republik Indonesia Nomor … Tahun … tentang Perlindungan Data Pribadi [The Draft of Personal Data Protection Bill Number … Year … of the Republic of Indonesia]. (2019). Kementerian Komunikasi dan Informatika Republik Indonesia [Ministry of Communication and Informatics of the Republic of Indonesia]. https://web.kominfo.go.id/sites/default/files/users/4752/Rancangan UU PDP Final %28Setneg 061219%29.pdf

Regmi, P. R., Waithaka, E., Paudyal, A., Simkhada, P., & van Teijlingen, E. (2016). Guide to the design and application of online questionnaire surveys. Nepal Journal of Epidemiology, 6(4), 640-644. https://doi.org/10.3126/nje.v6i4.17258

Revilla, M., Paura, E., & Ochoa, C. (2021). Use of a research app in an online opt-in panel: The Netquest case. Methodological Innovations, 14(1). https://doi.org/10.1177/2059799120985373

Sarkar, T., & Roychowdhury, S. (2019). Data wrangling with Python: Creating actionable data from raw sources. Packt Publishing. https://www.packtpub.com/product/data-wrangling-with-python/9781789800111

Seligman, M. (2018). PERMA and the building blocks of well-being. The Journal of Positive Psychology, 13(4), 333-335. https://doi.org/10.1080/17439760.2018.1437466

Shadiqi, M. A., Muluk, H., & Milla, M. N. (2018). Experiment replication: A proposed solution for developing psychological research in Indonesia. ANIMA Indonesian Psychological Journal, 33(4), 213-230. https://doi.org/10.24123/aipj.v33i4.1795

Sianipar, A., van Groenestijn, P., & Dijkstra, T. (2016). Affective meaning, concreteness, and subjective frequency norms for Indonesian words. Frontiers in psychology, 7:1907. https://doi.org/10.3389/fpsyg.2016.01907

Siaputra, I. B., Prawitasari, J. E., Hastjarjo, T. D., & Azwar, S. (2011). Subjective and projective measures of thesis writing procrastination: Real world and The Sims world. ANIMA Indonesian Psychological Journal, 26(2), 128-149. Available in Repositori Universitas Surabaya [Repository of University of Surabaya/UBAYA Repository] at http://repository.ubaya.ac.id/23845/

SmartSurvey. (n.d.). Skip logic surveys. SmartSurvey. Retrieved April 14, 2021 from https://www.smartsurvey.co.uk/skip-logic-surveys

Smith, A., de Salas, K., Lewis, I., & Schüz, B. (2017). Developing smartphone apps for behavioural studies: The AlcoRisk app case study. Journal of Biomedical Informatics, 72, 108-119. https://doi.org/10.1016/j.jbi.2017.07.007

Smith, D. (2003, January). Five principles for research ethics. Monitor on Psychology, 34(1), 56. https://www.apa.org/monitor/jan03/principles

StatCounter. (n.d.). Mobile operating system market share Indonesia. StatCounter: Global Stats. Retrieved March 14, 2021 from https://gs.statcounter.com/os-market-share/mobile/indonesia

Stephanie, C. (2020, November 27). Ini dia 5 penguasa pasar smartphone di Indonesia kuartal III-2020 [Here are 5 dominators of the Indonesian smartphone market in the third quarter of 2020]. Kompas. https://tekno.kompas.com/read/2020/11/27/09110097/ini-dia-5-penguasa-pasar-smartphone-di-indonesia-kuartal-iii-2020?page=all

Stockwell, K., Alabdulqader, E., Jackson, D., Basu, A., Olivier, P., & Pennington, L. (2019). Feasibility of parent communication training with remote coaching using smartphone apps. International Journal of Language and Communication Disorders, 54(2), 265-280. https://doi.org/10.1111/1460-6984.12468

Stoet, G. (2017). PsyToolkit: A novel web-based method for running online questionnaires and reaction-time experiments. Teaching of Psychology, 44(1), 24-31. https://doi.org/10.1177/0098628316677643

Stoycheff, E. (2016). Please participate in Part 2: Maximizing response rates in longitudinal MTurk designs. Methodological Innovations, 9. https://doi.org/10.1177/2059799116672879

SurveyMonkey. (n.d.a). SurveyMonkey: The world’s most popular free online survey tool. SurveyMonkey. Retrieved March 16, 2021 from https://www.surveymonkey.com/

SurveyMonkey. (n.d.b). Advanced branching. SurveyMonkey. Retrieved March 16, 2021 from https://help.surveymonkey.com/articles/en_US/kb/Advanced-Branching-Rules

SurveyMonkey. (n.d.c). Membuat riset skripsi lebih mudah dan cepat [Making undergraduate thesis research easier and faster]. SurveyMonkey. Retrieved March 16, 2021 from https://www.surveymonkey.de/mp/membuat-riset-skripsi-lebih-mudah-dan-cepat-dengan-surveymonkey/

SurveyMonkey. (n.d.d). Choose a plan that works for you. SurveyMonkey. Retrieved March 16, 2021 from https://www.surveymonkey.com/pricing/individual/?ut_source=pricing-teams-details

Susianto, H. (1992). Memadukan “ilmu” dan “seni” dalam menyusun skala Likert [Combining "science" and "art" in constructing a Likert scale]. Jurnal Psikologi Sosial, 1, 5-19.

Torous, J., Lipschitz, J., Ng, M., & Firth, J. (2020). Dropout rates in clinical trials of smartphone apps for depressive symptoms: A systematic review and meta-analysis. Journal of Affective Disorders, 263, 413-419. https://doi.org/10.1016/j.jad.2019.11.167

UK Data Service. (n.d.). Research data management. UK Data Service. Retrieved March 16, 2021 from https://www.ukdataservice.ac.uk/manage-data/legal-ethical/anonymisation.aspx

Varnhagen, C. K., Gushta, M., Daniels, J., Peters, T. C., Parmar, N., Law, D., Hirsch, R., Sadler Takach, B. S., & Johnson, T. (2005). How informed is online informed consent? Ethics & Behavior, 15(1), 37-48. https://doi.org/10.1207/s15327019eb1501_3

Woods, K. J. P., Siegel, M. H., Traer, J., & McDermott, J. H. (2017). Headphone screening to facilitate web-based auditory experiments. Attention, Perception, & Psychophysics, 79, 2064-2072. https://doi.org/10.3758/S13414-017-1361-2

Yu, S., Alper, H. E., Nguyen, A. -M., Brackbill, R. M., Turner, L., Walker, D. J., Maslow, C. B., & Zweig, K. C. (2017). The effectiveness of a monetary incentive offer on survey response rates and response completeness in a longitudinal study. BMC Medical Research Methodology, 17:77. https://doi.org/10.1186/s12874-017-0353-1

Zakiah, N. (2021, March 9). 7 brand HP yang kuasai pasar Indonesia, banyak yang punya: Adakah merk smartphone yang kamu pakai? [7 mobile phone brands dominating the Indonesian market, many pople have it: Is the brand of smartphone you use included?]. IDN Times. https://www.idntimes.com/tech/gadget/nena-zakiah-1/brand-smartphone-yang-menguasai-market-indonesia/7

Zakirman, Z. & Rahayu, C. (2018). Popularitas WhatsApp sebagai media komunikasi dan berbagi informasi akademik mahasiswa [The popularity of WhatsApp as undergratuate students' communication and academic information sharing media]. Shaut al-Maktabah: Jurnal Perpustakaan, Arsip dan Dokumentasi, 10(1), 27-38. https://www.rjfahuinib.org/index.php/shaut/article/view/7

Published
2022-01-31
How to Cite
Permatasari, I., & Susianto, H. (2022). Guidelines on Conducting Online Experiments with Repeated Measures and Using Smartphone Applications: [Panduan Melakukan Eksperimen Daring dengan Pengukuran Multipel serta Menggunakan Aplikasi Ponsel]. ANIMA Indonesian Psychological Journal, 37(1). https://doi.org/10.24123/aipj.v37i1.4470