Test Instrument Description
Autism spectrum disorder (ASD) presents a heterogeneous group of developmental disorders that are characterized by impairments in communication, social interaction, and the presence of limited, repetitive behaviors or interests. Currently, the prevalence of ASD has increased significantly, and, according to various researchers, it is at least 1.5% in developed countries (Langmann et al., 2017). In this regard, specialists of various directions, including psychologists, clinical psychiatrists, teachers, and others, are faced with the task of accurately diagnosing ASD. This paper provides the discussion of the Autism Diagnostic Observation Schedule (ADOS) as one of the most effective tools to diagnose ASD and determine its severity.
Test Instrument Authors and Purpose
The ADOS instrument was elaborated by an international group of specialists. The key authors are Catherine Lord, a clinical psychologist and Emeritus Professor of Psychology and Psychiatry at the University of California, Los Angeles School of Medicine and the University of Michigan, and Michael Rutter, a child psychiatrist, the founder of Child and Adolescent Psychiatry in the UK. Among other contributors, there are Katherine Gotham, Whitney Guthrie, Rhiannon J. Luyster, Pamela C. DiLavore, Somer L Bishop, and Susan Risi. The identified test is recommended by the UK’s National Institute for Health and Care Excellence (NICE) and the American Autism Association.
The ADOS is a standardized protocol for monitoring communicative and social behaviors that are indicative of ASD. This test instrument is beneficial in assessing communication, social interaction, playing, and the use of materials with the use of imagination in the presence of suspicion of autism in children (from 12 months) and adults of different ages, developmental levels, and speech skills. The interviewer creates standard situations using structured activities and method materials, in which behaviors important for diagnosing autism can appear (Langmann et al., 2017).
ADOS surveys can be carried out repeatedly in order to conduct differential diagnostics and determine clinical diagnosis and monitor the dynamics of the state and the success of the ongoing treatments. In addition, the results of the testing can be used in a further individual child care program. The test takes 40 to 60 minutes to complete, and an hour is required for a specialist to process the test results and provide a conclusion. Testing takes place in a specially equipped room, where other classes are not held.
Methods, Scores, Scale, Validity, and Reliability
The final report of ADOS contains a score to identify the severity of symptoms related to the ASD. It can be used by a psychotherapist to make a clinical diagnosis in combination with other information. A child’s final ADOS score reflects the depth of his or her autism and points to further actions that should be taken. The given test instrument has only one scale to measure a patientâs behaviors. It is assumed that a tested person is assigned one of the diagnostic groups, such as autism, the spectrum of autism, and beyond the spectrum of autism. The diagnostic group of autism differs from the group of the spectrum of autism by more pronounced manifestations of the disorder. This is the average value of such indicators as social interaction (facial expressions addressed to others, unusual eye contact, spontaneous initiation of joint attention, social responses, and the quality of social initiatives), imagination / creativity, as well as stereotypical forms of behavior and limited interests.
While ADOS was first published in 2003, its second edition, ADOS-2, became available in 2012. ADOS-2 involves five modules, each of which corresponds to a certain age and takes into account the extent of development and formation of speech. The modules differ by the degree of difficulty of tasks and offered games and the presence or absence of verbal speech tasks. The directions in the manual help a specialist in choosing the relevant module for a certain person.
Module 1 focuses on those children who cannot speak with phrases constantly. Module 2 applies to children who are able to practice phrasal speech, while they do not speak fluently, Module 3 is chosen for fluently-speaking children, and Module 4 is for fluently-speaking adolescents and adults. Non-speaking adolescents and adults compose the only group of persons with ASD, with whom the ADOS strategy cannot be used. A new Toddler Model targets children of 12-30 months to examine their phrase speech. Each of the modules is a sequence of game situations that are created by a specialist using a special set of toys and aids (Torres et al., 2020). As a result, the child receives a wide opportunity to express himself or herself in different conditions, on the basis of which it is possible to identify the characteristics of his or her development.
In a relaxed atmosphere of play and free interaction with a specialist, a child has to do anything but behave as he or she can. Although the illusion of direct communication and freedom for the child is created, the specialist follows the rules of behavior that are strictly prescribed in the test manual, where words and gestures are not accidental. Accordingly, the child is immersed in situations where he or she may exhibit certain forms of behavior that have differential diagnostic value in terms of disorders on the autism spectrum. During processing, the specialist translates the recorded behavioral features into digital indicators.
The algorithms of both ADOS and ADOS-2 demonstrate high reliability and predictive validity, which characterizes the degree of accuracy in determining autism spectrum disorders. It is confirmed by the data of the study obtained on a large sample (over 1000 people) for all modules (Randall et al., 2018). It is a reliable diagnostic tool recognized as objective in vivo and demonstrates high predictive reliability. The systematic review of 12 scholarly articles on the accuracy of ADOS found that its overall sensitivity score is s 0.94 (95% CI 0.89 to 0.97) (Randall et al., 2018). The specificity score was 0.80 (95% CI 0.68 to 0.88), which shows a high rate of validity (Randall et al., 2018).
However, it should also be stated that children with different psychological disorders can be erroneously diagnosed as autism positive. For example, children with schizophrenia or those having attention deficit hyperactivity disorder (ADHD) are at risk of false positive outcomes.
Although ADOS is highly specific for symptoms associated with ASD, its results do not replace the need for clinical evaluation and formal diagnosis. Combining ADOS-2 and the Autism Diagnostic Interview-Revised (ADI-R) improves accuracy as it is considered to be the gold standard for diagnosing autism. It is important to state that the results of this test alone cannot make an accurate diagnosis. It is advisable to undergo a comprehensive examination, paying attention to different testing, family history and observation, as well as behavioral changes.
Using and Improving the ADOS
Personally, I would use ADOS as an evaluation tool since the timely identification of violations allows for a more accurate way to accompany a person with ASD. It also increases the effectiveness of interventions and improves future prognosis and adaptation. If the parents or people around the child are in the slightest doubts about the child’s behavior, they need to refer to a specialist. According to the observations of scholars, if intensive work with a child with autism spectrum disorders is carried out from 2 to 5 years old, the prognosis is usually positive (Langmann et al., 2017). To clarify the diagnosis, I would ask the parents of the child visit a clinical psychologist.
Another benefit of ADOS is that by offering standardized metrics and materials, it helps in diagnosing ASD across languages. Since it applies to different children and adults, it is a cost-effective addition to any hospital, clinic, or school that serves people with developmental disabilities. However, the quality of using ADOS largely depends on the interviewer’s experience in applying this technique and his clinical skills. Therefore, before using ADOS, the interviewer must undergo special training in a clinical or research team (Torres et al., 2020). Mandatory and usually paid training in ADOS-diagnostics, as well as the costs of test materials, can impose additional restrictions on the use of this diagnostic method.
A significant limitation of diagnostics using ADOS is the lack of the possibility of its use for examining non-speaking adolescents and adults. In this regard, it is possible to recommend using ADOS with other available tests. Since ADOS provides information based on short-term observation for social forms of behavior and communication only at a certain point in time, in combination with it for a more objective and comprehensive diagnosis, it is recommended to use a detailed interview, such as ADI-R. As the test instrument, ADOS can be improved by continuous updating in accordance with the growing body of research. The latest recommendations from autism organizations and scholarly articles should be integrated into the ADOS.
Considering the mentioned benefits and limitation of using ADOS, I would improve the training of specialists, who conduct this testing. Namely, clinical specialists and teachers should be aware of the signs and behaviors that require autism diagnostics. It is essential to provide training sessions at least twice a year, thus ensuring that the process and results of testing would be accurate and non-biased. As for procedures, in the process of performing specific tasks or immediately after their completion of the test, notes should be taken regarding behaviors (for example, a smile or voluntary communication of information). All these records should be taken into account in encryption, which would ensure comprehensive examination.
Thus, the ADOS test serves as a standardized observation of behavior during specially organized play. After the diagnosis, coding is carried out, and a careful scoring of points for various parameters and the conclusion of the diagnosis are provided. The use of this test instrument implies choosing one of the five modules, depending on the age and abilities of the tested child. The results allow for identifying the severity of the disease and clarifying further actions for the clinical specialists and parents. The combined use with ADI-R, better training of specialists, and greater attention to notes are recommended as potential improvements.
Instrument Development for Leadership Ability Assessment
People who compose human capital are the most important economic resource of any company or organization. They serve as the source of income, competitiveness, and overall success, which is due to the growing rivalry, in which the most competitive and efficient companies find ways to resolve dynamic contradictions (Sari & Bashori, 2020). In this case, the views and actions of leaders become critical, and it prioritizes the need to recognize and develop leadership skills and competencies. This paper develops an instrument that is aimed to assess managersâ leadership ability based on the combination of a Myers-Briggs Type Indicator (MBTI), 360 Degree Feedback, and extraordinary task performance.
Instrumentâs Essence, Construct, Sources, and Process
Considering the available leadership ability testing instruments, it becomes evident that each of them focuses on one or several areas, such as personality or behaviors. It should be stated that leadership is understood as a form of manifestation of influence on people’s behavior, which is based on predominantly socio-psychological contacts (Sari & Bashori, 2020). It is significant in achieving the goals of an organization by harmonizing the interests of the company and the followers of the leader to establish a synergistic effect of effective collaboration.
The review of the literature on the most widespread tests and their application allows for assuming that there is a need to develop a more comprehensive tool that would address leadership competencies from different perspectives. Therefore, this paper presents an instrument that integrates the Myers-Briggs Type Indicator (MBTI), extraordinary task performance, and 360 Degree Feedback.
The MBTI was developed in the 1940s by Catherine Briggs and Isabelle Briggs-Myers, the American researchers. This instrument is a diagnostic and a typological system created by the authors based on the theory of psychological types by Carl Jung (Sari & Bashori, 2020). MBTI describes personality preferences and the extent of their presence in a personality. The developed instrument will include 40 questions with two to three choices to answer (10 questions for each of four scales). The first scale will be related to how a person manifests and recovers energy: extraversion (E) versus introversion (I). Extroverts prefer to restore their strength through contact with objects of the outside world, with other people, in activities.
Introverts prefer restoring strength through contact with your inner world of ideas, experiences, and impressions. The second scale will test the ways of collecting information about the world: intuition (K) versus sensors (S). The third scale will characterize the ways of making decisions: thinking based on logic (T) versus feeling (F), relying on values and emotions. Ultimately, the fourth scale – preference for organization, planning as opposed to flexibility (P) versus spontaneity (P).
Another dimension of the developing instrument is evaluating a potential leaderâs ability to think and act in extraordinary conditions. During the assessment, a tester would offer a random case that should be resolved by the person, using his or her creative and critical thinking. In this regard, attention should be paid to the function of logic as an orientation towards objective reality, production processes, and the achievement of professionalism. The approach with which the potential leader follows ethics should also be assessed. Namely, a specialist would consider a focus on the world of emotions and relationships between people, the ability to choose the right psychological distance, and dependence on the opinions of others.
A component of 360 Degree Feedback is one more important aspect that will collect and evaluate the views of colleagues, managers, and other team members. Effective feedback shows the current state of affairs and provides a glimpse into the future without lingering in the past.
Feedback is ineffective if it inflates or hurts someone’s ego. Finding a middle ground can be difficult, and a specialist needs to keep a balance between explanation and teaching, subtleties and embellishment, and impartiality and personal accusations (Das & Panda, 2017). To identify the qualities that contribute to or hinder the perception of a person as a leader, the semi-structured interviews will use the following questions: What are the characteristics of leadership does the person have?, What qualities does your team need?, What weaknesses and strengths can you name?, et cetera. With the help of these questions, it is expected to reveal the attitude of subordinates to their potential leader.
The process of developing the above test instrument began with the investigation of the related academic articles and websites that offer online testing or some details about leadership tests. It was found that the majority of these tests are related to a single dimension of leadership, while it was suggested that it is impossible to hope that one tool will show everything at once. For example, a survey will reveal only leadership potential, not whether the employee is an actual leader in practice. In cases and stress interviews, people only imagine them in a difficult situation, but in reality, they are usually in a calm atmosphere. It means that in reality, they can act differently than in the simulated situation. Therefore, it was decided to combine the most convincing elements of testing that can help to assess a personâs leadership ability in a multifaceted manner.
The resources used to develop the test instrument that is identified above included the broad review of the academic literature to understand the current state of affairs in leadership testing. A range of articles that were directly used to develop the instrument is cited in the references section. In addition, the reliable websites were accessed to review the structure of the Myers-Briggs Type Indicator (MBTI) and adapt it accordingly. The literature on validity, reliability, and pilot testing was also examined to discuss them in relation to the developed test instrument.
Validity, Reliability, and Pilot Testing
The purpose of the identified instrument is to reveal leadership ability among those employees who already work in a company. Its primary idea is to find those people who can be successful leaders, using their logical yet creative thinking. At the same time, the existing leaders can also be assessed using this method to either confirm or reject their leadership ability. In some cases, it is better for companies to change leaders and encourage employeesâ initiatives that can be beneficial in terms of the rapidly developing and changing markets. Accordingly, the age range of those who can be enrolled in the developed test, the target population, is not limited.
An essential factor in the choice of a particular method or test is its validity. In this case, it is understood as the compliance with what it is intended for and determining exactly what it should. The validity of a test will be determined by the correlation of its results with other criteria of the measured property (Mohajan, 2017). Namely, the validity of the identified instrument will be stated by the correlation of test results with the success of the corresponding activity.
The reliability of the test is caused by the consistency of the results obtained with its help. The reliability of the test will be determined by establishing correlations between the results of the first and repeated use of the test or by comparing the data obtained during the test with the results of applying an equivalent test (Mohajan, 2017). To ensure reliability, it will be established that the test allows receiving the same results regardless of the number of respondents and the contingent.
Pilot testing implies some preliminary actions and preparation to verify the test approach on a small number of participants. While pilot testing is an optional step, it can save time and resources if some drawbacks would be revealed (Mohajan, 2017). Since the correct design of a test instrument is a prerequisite for conducting it, it is critical to take the time to evaluate, test, and refine the design before launching it. By doing this, one can ensure that the tool runs smoothly and that proper results would be obtained as a result. It is expected to identify possible weak points of the developed test instrument and eliminate them before using it in real-life conditions.
Among other anticipated outcomes, there are the benefits of combining the questionnaire, semi-structured interviews, and performance evaluation. Potential leaders receive the opportunity to show their leadership ability from different perspectives. In general, pilot testing will allow approving the wording of the tasks, estimate the time of the sessions, and shows additional reference points for further improvement.
In the given case, pilot testing will be organized based on the subsequent application of all the three components. First, Myers-Briggs Type Indicator (MBTI) will be offered to the study participants; second, 360 Degree Feedback component will be implemented by asking the colleagues of the participants to characterize them anonymously; and third, extraordinary task performance will be conducted (Das & Panda, 2017).
It is in non-standard, stressful situations, in which the leadership qualities of a leader are to be manifested. Also, any team game will help to understand how an employee interacts with the team, and it is useful to observe how the manager interacts with other players. If a person takes the role of a coordinator and brings the team to the winning, these are leadership competencies, such as the ability to create a team of like-minded people and the ability to convince. For example, the leader can start reciting poetry, singing, dancing, et cetera. The essence of testing is that a person will not be confused with non-standard questions and requests.
To conclude, this paper discusses the development of a new test instrument to identify and characterize the leadership ability of leaders and those employees who can potentially become leaders. The offered instrument presents the integration of three components, including 360 Degree Feedback, the Myers-Briggs Type Indicator (MBTI), and extraordinary task performance. These components were selected to ensure that leadership ability testing is comprehensive, allowing candidates to show their critical and creative thinking, actions, and attitudes of others towards them. The paper also provides the discussion of the test validity, reliability, and pilot testing.
References
Das, U. K., & Panda, J. (2017). The impact of 360 Degree Feedback on employee role in leadership development. Asian Journal of Management, 8(4), 962-966.
Langmann, A., Becker, J., Poustka, L., Becker, K., & Kamp-Becker, I. (2017). Diagnostic utility of the autism diagnostic observation schedule in a clinical sample of adolescents and adults. Research in Autism Spectrum Disorders, 34, 34-43.
Mohajan, H. K. (2017). Two criteria for good measurements in research: Validity and reliability. Annals of Spiru Haret University. Economic Series, 17(4), 59-82.
Randall, M., Egberts, K. J., Samtani, A., Scholten, R. J., Hooft, L., Livingstone, N., SterlingâLevis, K., Woolfenden, S., & Williams, K. (2018). Diagnostic tests for autism spectrum disorder (ASD) in preschool children. Cochrane Database of Systematic Reviews, 24(7), 1-113.
Sari, E. Y. D., & Bashori, K. (2020). Predicting managerial styles: Is the Myers-Briggs Type Indicator still useful? Journal of Education and Learning (EduLearn), 14(4), 617-622.
Torres, E. B., Rai, R., Mistry, S., & Gupta, B. (2020). Hidden aspects of the research ADOS are bound to affect autism science. Neural Computation, 32(3), 515-561.