PROTEA-M: MULTIMODAL FRAMEWORK FOR INCLUSIVE COMMUNICATION IN FLOOD SITUATIONS
Abstract
Early warning systems are the main tool for notifying and guiding the population in emergency situations; however, they still rely on standardized messages that do not consider cognitive and communicative differences. This limitation directly affects individuals with Autism Spectrum Disorder, who require structured and predictable forms of information organization for understanding and decision-making in risk contexts. This study presents the PROTEA-M framework (Program for Readiness and Organized Training in Emergencies for Autistic Individuals - Multimodal), a multimodal predictive-generative model designed to produce inclusive content for preparedness in flood scenarios. The model estimates the communicative profile and intervention plan and, based on this estimation, generates personalized Social Stories. The predictive module employs an ensemble of Extreme Learning Machines with evidence fusion and probabilistic calibration, achieving superior performance compared to baselines, with accuracy of 0.6923 and ROC-AUC of 0.9782, in addition to improved probabilistic quality. The generative module produces multimodal narratives using language models, translating clinical variables into structured visual and textual sequences. PROTEA-M contributes to the advancement of early warning systems by incorporating communicative personalization and cognitive accessibility.
Author Biographies
Flutter mobile developer, specialized in responsive design and application integration using REST APIs. Bachelor's degree in Computer Engineering from Universidade Paulista and Master's degree in Computer Engineering from the University of Pernambuco. Certified as a Professional Scrum Master by Scrum.org and as a Flutter Developer by AndroidATC.
Graduated in Electrical Engineering from the Federal University of Pernambuco (1978), Master's degree in Electrical Engineering from the Federal University of Pernambuco (1995), and Ph.D. in Computer Science from the Center for Informatics of the Federal University of Pernambuco (2007). Associate Professor II at the Catholic University of Pernambuco and at the Polytechnic School of the University of Pernambuco.
References
ABDULLAH, Maria et al. Artificial intelligence-based assistive technologies for autism spectrum disorder: a systematic review. Journal of Autism and Developmental Disorders, v. 52, n. 9, p. 4029-4050, 2022.
AHMED, S. et al. An optimized Kernel Extreme Learning Machine for the classification of the autism spectrum disorder by using gaze tracking images. Applied Soft Computing, v. 120, p. 108654, 2022. DOI: https://doi.org/10.1016/j.asoc.2022.108654
BALOG-WAY, Dominic H. P.; MCCOMAS, Katherine A.; BESLEY, John C. The evolving field of risk communication. Risk Analysis, v. 40, n. S1, p. 2240-2262, 2020. DOI: https://doi.org/10.1111/risa.13615
BASHER, Reid. Global early warning systems for natural hazards: systematic and people-centred. Philosophical Transactions of the Royal Society A, v. 364, n. 1845, p. 2167-2182, 2006. DOI: https://doi.org/10.1098/rsta.2006.1819
BEUKELMAN, David R.; MIRENDA, Pat. Augmentative and alternative communication: supporting children and adults with complex communication needs. 4. ed. Baltimore: Paul H. Brookes Publishing, 2013.
BOGDASHINA, Olga. Sensory perceptual issues in autism and Asperger syndrome: different sensory experiences, different perceptual worlds. 2. ed. London: Jessica Kingsley Publishers, 2016.
BREIMAN, Leo. Random forests. Machine Learning, v. 45, n. 1, p. 5-32, 2001. DOI: https://doi.org/10.1023/A:1010933404324
BRIER, Glenn W. Verification of forecasts expressed in terms of probability. Monthly Weather Review, v. 78, n. 1, p. 1-3, 1950. DOI: https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2
BROWN, Tom et al. Language models are few-shot learners. Advances in Neural Information Processing Systems, v. 33, p. 1877-1901, 2020.
CENTERS FOR DISEASE CONTROL AND PREVENTION (CDC). Data and statistics on autism spectrum disorder. Atlanta: CDC, 2023. Disponível em: https://www.cdc.gov/autism/data-research/index.html Acesso em: 25 maio 2026.
CERDA, Patricio; VAROQUAUX, Gaël; KÉGL, Balázs. Similarity encoding for learning with dirty categorical variables. Machine Learning, v. 107, p. 1477-1494, 2018. DOI: https://doi.org/10.1007/s10994-018-5724-2
COHEN, Jacob. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, v. 20, n. 1, p. 37-46, 1960. DOI: https://doi.org/10.1177/001316446002000104
CORTES, Corinna; VAPNIK, Vladimir. Support-vector networks. Machine Learning, v. 20, p. 273-297, 1995. DOI: https://doi.org/10.1007/BF00994018
COVER, Thomas; HART, Peter. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, v. 13, n. 1, p. 21-27, 1967. DOI: https://doi.org/10.1109/TIT.1967.1053964
DIETTERICH, Thomas G. Ensemble methods in machine learning. In: MULTIPLE CLASSIFIER SYSTEMS. Berlin: Springer, 2000. p. 1-15. DOI: https://doi.org/10.1007/3-540-45014-9_1
ELDAWLATLY, Abdelazeem. Writing the methods section. Saudi Journal of Anaesthesia, v. 13, supl. 1, p. S10-S11, 2019. DOI: https://doi.org/10.4103/sja.SJA_685_18
FAWCETT, Tom. An introduction to ROC analysis. Pattern Recognition Letters, v. 27, n. 8, p. 861-874, 2006. DOI: https://doi.org/10.1016/j.patrec.2005.10.010
FAYYAD, Usama; PIATETSKY-SHAPIRO, Gregory; SMYTH, Padhraic. From data mining to knowledge discovery in databases. AI Magazine, v. 17, n. 3, p. 37-54, 1996. DOI: https://doi.org/10.1609/aimag.v17i3.1230
FENG, Yi et al. SS-GEN: a social story generation framework with large language models. In: Proceedings of the AAAI Conference on Artificial Intelligence. v. 39, n. 2, p. 1300-1308, 2025.
FRITH, Uta. Autism: explaining the enigma. Oxford: Blackwell, 2003.
GRAY, Carol. The new social story book. 10. ed. Arlington: Future Horizons, 2010.
GUO, Chuan et al. On calibration of modern neural networks. In: International Conference on Machine Learning. [S.l.]: PMLR, 2017. p. 1321-1330.
HASTIE, Trevor; TIBSHIRANI, Robert; FRIEDMAN, Jerome. The elements of statistical learning. 2. ed. New York: Springer, 2009.
HOERL, Arthur E.; KENNARD, Robert W. Ridge regression: biased estimation for nonorthogonal problems. Technometrics, v. 12, n. 1, p. 55-67, 1970. DOI: https://doi.org/10.1080/00401706.1970.10488634
HUANG, Guang-Bin; ZHU, Qin-Yu; SIEW, Chee-Kheong. Extreme learning machine: theory and applications. Neurocomputing, v. 70, n. 1-3, p. 489-501, 2006. DOI: https://doi.org/10.1016/j.neucom.2005.12.126
INSTITUTO BRASILEIRO DE GEOGRAFIA E ESTATÍSTICA (IBGE). Censo Demográfico 2022. Rio de Janeiro: IBGE, 2022. Disponível em: https://www.ibge.gov.br Acesso em: 25 maio 2026.
INTERGOVERNMENTAL PANEL ON CLIMATE CHANGE (IPCC). Climate change 2023: synthesis report. Geneva: IPCC, 2023. Disponível em: https://www.ipcc.ch/report/ar6/syr/ Acesso em: 25 maio 2026.
KENT, Rachel et al. Preparing for emergencies among families of children with autism spectrum disorder. Journal of Autism and Developmental Disorders, v. 43, n. 3, p. 666-675, 2013. DOI: https://doi.org/10.1007/s10803-012-1618-0
KOHAVI, Ron. A study of cross-validation and bootstrap for accuracy estimation and model selection. In: International Joint Conference on Artificial Intelligence. San Francisco: Morgan Kaufmann, 1995. p. 1137-1145.
KOHONEN, Teuvo. Self-organizing maps. 3. ed. Berlin: Springer, 2001.
KOKINA, Anastasia; KERN, Lee. Social story interventions for students with autism spectrum disorders: a meta-analysis. Journal of Autism and Developmental Disorders, v. 40, n. 7, p. 812-826, 2010. DOI: https://doi.org/10.1007/s10803-009-0931-0
LINDELL, Michael K.; PERRY, Ronald W. The protective action decision model: theoretical modifications and additional evidence. Risk Analysis, v. 32, n. 4, p. 616-632, 2012. DOI: https://doi.org/10.1111/j.1539-6924.2011.01647.x
LUNDGREN, Regina E.; MCMAKIN, Andrea H. Risk communication: a handbook for communicating environmental, safety, and health risks. 6. ed. Hoboken: Wiley, 2018.
LUNDBERG, Scott M.; LEE, Su-In. A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems, 2017.
MICCI-BARRECA, Daniele. A preprocessing scheme for high-cardinality categorical attributes in classification and prediction problems. ACM SIGKDD Explorations Newsletter, v. 3, n. 1, p. 27-32, 2001. DOI: https://doi.org/10.1145/507533.507538
MILETI, Dennis S.; SORENSEN, John H. Communication of emergency public warnings. Oak Ridge: ORNL, 1990.
NAEINI, Mahdi Pakdaman; COOPER, Gregory; HAUSKRECHT, Milos. Obtaining well calibrated probabilities using Bayesian binning. In: AAAI Conference on Artificial Intelligence. 2015. p. 2901-2907.
PEEK, Lori; STOUGH, Laura. Children with disabilities in the context of disaster: a social vulnerability perspective. Child Development, v. 81, n. 4, p. 1260-1270, 2010. DOI: https://doi.org/10.1111/j.1467-8624.2010.01466.x
QUINLAN, J. Ross. Induction of decision trees. Machine Learning, v. 1, p. 81-106, 1986. DOI: https://doi.org/10.1007/BF00116251
REIMERS, Nils; GUREVYCH, Iryna. Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: Conference on Empirical Methods in Natural Language Processing. Hong Kong: ACL, 2019. p. 3982-3992. DOI: https://doi.org/10.18653/v1/D19-1410
ROBERTSON, Caroline E.; BARON-COHEN, Simon. Sensory perception in autism. Nature Reviews Neuroscience, v. 18, n. 11, p. 671-684, 2017. DOI: https://doi.org/10.1038/nrn.2017.112
ROWLAND, Charity. Matriz de Comunicação. Portland: Oregon Health & Science University, 2011.
RUTHERFORD, Marion et al. Visual supports at home and in the community for autistic children: a scoping review. Autism, v. 24, n. 7, p. 1737-1752, 2020. DOI: https://doi.org/10.1177/1362361320901572
SAITO, Takaya; REHMSMEIER, Marc. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLOS ONE, v. 10, n. 3, e0118432, 2015. DOI: https://doi.org/10.1371/journal.pone.0118432
SHAH, Ashfaq Ahmad et al. Community social barriers to non-technical aspects of flood early warning systems and NGO-led interventions: the case of Pakistan. Frontiers in Earth Science, v. 11, p. 1068721, 2023. DOI: https://doi.org/10.3389/feart.2023.1068721
STOUGH, Laura M.; KELMAN, Ilan. People with disabilities and disasters. New York: Palgrave Macmillan, 2018.
SUTTON, Jeannette; TIERNEY, Kathleen. Disaster preparedness: concepts, guidance, and research. Boulder: University of Colorado, 2006.
TEST, David W. et al. Evidence-based practices for individuals with autism. Exceptional Children, v. 78, n. 2, p. 135-155, 2011. DOI: https://doi.org/10.1177/001440291107800201
UNITED NATIONS CHILDREN'S FUND (UNICEF). Disability-inclusive disaster risk reduction. New York: UNICEF, 2023.
UNITED NATIONS OFFICE FOR DISASTER RISK REDUCTION (UNDRR). Global assessment report on disaster risk reduction 2022. Geneva: UN, 2022.
UNITED NATIONS OFFICE FOR DISASTER RISK REDUCTION (UNDRR). Sendai framework for disaster risk reduction 2015-2030. Geneva: UN, 2015.
VARMA, Sudhir; SIMON, Richard. Bias in error estimation when using cross-validation for model selection. BMC Bioinformatics, v. 7, n. 91, 2006. DOI: https://doi.org/10.1186/1471-2105-7-91
VASWANI, Ashish et al. Attention is all you need. Advances in Neural Information Processing Systems, v. 30, 2017.
WORLD BANK. Inclusive early warning systems: a lifeline for all. Washington: World Bank, 2022.
WORLD METEOROLOGICAL ORGANIZATION (WMO). State of the global climate 2021. Geneva: WMO, 2021.
License
Copyright (c) 2026 RECIMA21 - Revista Científica Multidisciplinar - ISSN 2675-6218
This work is licensed under a Creative Commons Attribution 4.0 International License.
Os direitos autorais dos artigos/resenhas/TCCs publicados pertecem à revista RECIMA21, e seguem o padrão Creative Commons (CC BY 4.0), permitindo a cópia ou reprodução, desde que cite a fonte e respeite os direitos dos autores e contenham menção aos mesmos nos créditos. Toda e qualquer obra publicada na revista, seu conteúdo é de responsabilidade dos autores, cabendo a RECIMA21 apenas ser o veículo de divulgação, seguindo os padrões nacionais e internacionais de publicação.