OPERATIONAL SUPPORT
Standard Setting
For almost every assessment program, at some point, a single or multiple cut points must be determined in order to make critical pass/fail or proficiency decisions. ACS staff are experts in all aspects of standard setting and will take the time to fully understand the nature of the assessment program and its candidates, and the precise nature of how and when the cut scores will be implemented. Once the most appropriate methods for conducting the standard setting, ACS can handle all aspects of the project and will deliver a comprehensive report supporting the validity and fairness of the cut score recommendations.
Outsourced Psychometric Staff
Psychometrics are a key element in the continual development and maintenance of any assessment. For many programs, the ideal arrangement is to work with external psychometric resources for program maintenance, oversight, and evaluation. ACS staff have worked with programs – from small to large – to provide psychometric direction and operational support for their assessments.
Alignment
A critical component of any assessment program is the alignment of the key elements (e.g., content specification, content, opportunity to learn). Although it is often assumed that appropriate test development will automatically result in alignment, many programs have realized the benefit of a formal alignment review to identify the strengths and areas in need of improvement. ACS has conducted numerous independent alignment studies and conducted research on this topic.
Psychometric Analysis
An essential component of every assessment program is the psychometric foundation of the program. These foundations can be established through fairly simple item analyses and item calibration procedures that will help ensure that all items on the assessment are functioning as expected. It can also be established through the psychometric equating or linking of multiple forms of the assessment, ensuring that all candidates who take each test for are treated fairly and equitably. At ACS, we are committed to not only delivering the highest quality psychometric analyses but also to fully explaining the analyses to our clients, including the important insights that can be derived while reviewing the results.
Technical Documentation
Documentation of assessment design, development, and maintenance efforts is critical to maintaining the defensibility of a testing program and for ongoing evaluation. However, determining what and how to document can be challenging and many programs have sought external assistance to maintain accurate documentation. ACS staff have experience in creating such documentation for both internal and external purposes.
