Valdez, L. (2002). Performance-based assessment: promoting achievement for English language learners.

ERIC/CLL news bulletin, 26(1), 1-3

n e w s b u l let i n
ERIC/CLL
CLEARINGHOUSE ON LANGUAGES AND LINGUISTICS CENTER FOR APPLIED LINGUISTICS Volume 26, No 1, Fall 2002

Performance-Based Assessment: Promoting Achievement for English Language Learners

Lorraine Valdez Pierce, George Mason University

Focus on Accountability
In December 2001, President George W. Bush signed into law the No Child Left Behind Act (NCLB). The Act has as its stated purpose the improvement of the educational achievement of economically disadvantaged and minority children, including those who are learning English as an additional language, or English language learners (ELLs). This legislation aims to close the achievement gap between high- and low-performing children, especially between minority and non-minority students (No Child Left Behind Act, 2001). It focuses on ensuring that schools are held accountable for helping all students meet state standards. It also places increasing pressure on schools and districts to include as many students as possible, including ELLs and students with disabilities, in large-scale assessment programs. States are required to report test scores for ELLs as a group and to use test results to improve the educational attainment of these students. By academic year 2005-2006, schools will be held accountable for, at a minimum, annual testing of all children in Grades 3-8 in reading or language arts and mathematics. In the interim, annual testing is to be conducted at various grade ranges, and schools failing to meet improvement goals for two consecutive years beginning in 2003-2004 are to be identified for sanctions, such as replacing teachers and providing students the option of transferring to another school. Standardized tests used in large-scale assessment programs are supposed to measure a representative sample of knowledge defined by state and local standards and curricula. To some extent, and for some students, these tests may provide evidence of school learning. But for ELLs in U.S. public schools, standardized test results are also likely to reflect limited proficiency in English and a lack
4646 40TH STREET NW

of opportunity to learn the subject matter of the tests (Amrein & Berliner, 2002; Calkins, Montgomery, & Santman, 1998; Heubert & Hauser, 1999; Kohn, 2000; McKeon, 1994; OMalley & Pierce, 1994; Stiggins, 2002). Are current approaches to assessment improving learning for ELLs? How can we help ELLs reach the point where standardized tests can be used as a valid gauge of their achievement? What is the role of classroom-based assessment in preparing ELLs to take standardized tests? This article examines the role of classroom-based assessment, and of performance-based assessment in particular, in promoting learning for ELLs in schools that are increasingly under pressure to prepare these students to pass high-stakes, standardized tests.

Defining the Ultimate Goal of Schooling

Schooling has been described as having at least three goals: education, training, and learning (Amrein & Berliner, 2002). Education is the broadest and the hardest to measure, with generalizability or transfer of learning to new situations and tasks being a central characteristic. Training refers to a narrow form of learning, where transfer of learning is measured on tasks that are highly similar to those used in the training (Amrein & Berliner, p. 10). Examples of training are tasks such as naming the presidents or using a map key. Learning, on the other hand, is the process through which students apply knowledge beyond basic facts and procedures. Examples of learning would be writing descriptive paragraphs and engaging in demonstrations, analyses, and justifications. Education can be defined, then, as the transfer of learning, that is the application of what is learned in one domain or context to that of another domain or context (Amrein & Berliner, 2002, p.10).
(202) 362-0700

Similarly, Marzano, Pickering, & McTighe (1993) propose that, Although acquiring content knowledge is important, it is perhaps not the most important goal of education. Ultimately, developing mental habits that will enable individuals to learn on their own whatever they want or need to know at any point in their lives is probably the most important goal of education (p. 3). Gardner (1999), too, considers mental habits, or thinking and inquiry processes, to be important goals of schooling. He proposes that students learn by probing a small set of examples from the disciplines, rather than by covering a broad range of topics in much less depth, and by discussing and conducting projects, with the ultimate goal being the ability to transfer learning to a wide range of tasks. The mental habits that develop from this type of study should help students develop the skills and abilities needed for life-long learning and for success in life, such as the ability to think and analyze; locate information; work collaboratively on teams; become problem solvers; and perform real-world tasks (Darling-Hammond, Ancess, & Falk, 1995). Transfer of learning and the development of mental habits that facilitate that transfer, then, are worthy goals of schooling. But what types of assessments are able to capture or promote progress toward these goals?

Classroom-Based Assessments
Research has shown that improved assessment practices at the classroom level can have powerful, beneficial effects on transfer of learning and measures of achievement, including standardized test scores (Black & Wiliam, 1998; Stiggins, 2002). In fact, Black &

inside
4 4 5 6 New ERIC/CLL Products Partner News News From CAL New Standards for Foreign Language Teaching: A Vision for the Future
continued on page 2

W A S H I N G T O N D C 2 0 0 1 6 -1859

( 8 0 0 ) 2 7 6 - 9 8 3 4 W W W . C A L . O R G /ERICCLL

Performance-Based Assessment
continued from page 1

Wiliam, in a review of over 250 articles, found that improved formative or classroom assessment practices helped low achievers more than other students. This revealing finding has direct implications for NCLB and for school systems that want to close the achievement gap. To make improvements, however, teachers must be provided with the assessment tools they need for increasing the achievement of ELLs. New understandings of the learning process indicate that assessment and learning are intimately linked. These new understandings of learning need to be applied to classroom-based assessment practices (Marzano, Pickering, & McTighe, 1993). Among these practices, performance-based assessment appears to hold promise for improving the educational attainment of ELLs.

or generalizability of learning from facts and procedures to applications in meaningful contexts. A large range and number of tasks are needed over time, however, to ensure the generalizability of PBAs. Can performance-based assessments be used to monitor and support the learning of ELLs? A number of factors make PBAs more appropriate for ELLs than traditional testing formats (Frisby, 2001; Hamayan & Damico, 1991; OMalley & Pierce, 1996). Well-constructed performance tasks are more likely than traditional types of assessment to do the following: provide comprehensible input to students use meaningful, naturalistic contextembedded tasks through hands-on or collaborative activities show what students know and can do through a variety of assessment tasks support the language and cognitive needs of ELLs allow for flexibility in meeting individual needs use criterion-referenced assessment for judging student work provide feedback to students on strengths and weaknesses generate descriptive information that can guide instruction provide information for teaching and learning that results in improved student performance Further, PBAs have the potential to provide in-depth information about a students ability to integrate knowledge for specific curriculum objectives or standards. Teachers using PBAs in the classroom have three types to choose from: products, performances, or process-oriented assessments. (McTighe & Ferrara, 1998). Products are works produced by students that provide concrete examples of their application of knowledge, for example, writing samples, projects, art or photo exhibits, and portfolios. Performances allow students to demonstrate application of their knowledge and skills under the direct observation of the teacher. Students may engage in tasks that are useful outside of school, such as asking for directions by telephone, demonstrat-

Using Performance-Based Assessment to Promote Learning

Classroom-based assessments may be of two broad types: selected-response and constructed-response formats. Selected-response formats provide response items for students to choose from (such as multiple-choice, true-false, and matching items). Constructedresponse formats, on the other hand, ask students to develop a response, create a product, or conduct a demonstration (Feuer & Fulton, 1993; Frisby, 2001; Herman, Aschbacher, & Winters, 1992; McTighe & Ferrara, 1998). These types of assessments allow more than one correct answer to a problem and typically involve higher-order thinking skills. Performance-based assessment (PBA), which uses a constructed-response format, has as its primary purpose the improvement of learning. Performancebased assessment links assessment to instruction through the use of meaningful and engaging tasks. Performance tasks may also call for integration of language and content-area skills. Authentic assessment, a type of PBA, promotes application of knowledge and skills in situations that closely resemble those of the real world (Frisby, 2001; McTighe & Ferrara, 1998; Wiggins, 1998). Authentic assessments are potentially more motivating than other types because they engage students in realistic uses of language and contentarea concepts. Authentic assessment and other types of PBA can be used in the service of education to promote transfer

ing a process, or arguing a position. All of these can demand high levels of language skill. Examples of performance tasks include oral reports, skits and roleplays, demonstrations, and debates. Process-oriented assessments provide insight into student thinking, reasoning, and motivation. They can provide diagnostic information on how well students use learning strategies and may lead to independent learning when students are asked to reflect on their learning and set goals to improve it. Some examples of process-oriented assessments are think-alouds, selfassessment checklists or surveys, learning logs, and individual or pair conferences. Products, performances, and processoriented assessments can all be used to generate rich information on ELLs ability to transfer learning and meet state and local standards. Two features of performance-based assessment help support the development of mental habits that lead to independent learning. The first is referred to as visible criteria. A fundamental tenet of performance-based assessment is the sharing of standards and making the criteria for evaluation visible to students. Teachers share their expectations for student work and performance in as explicit terms as possible, using a scoring rubric, checklist, or other assessment tool and representative samples of student work. This approach is especially important with ELLs, who have been shown to benefit from the teachers sharing of the assessment criteria in advance of the assessment itself (Kolls, 1992). When teachers state expectations for learning in terms of specific outcomesin language the students can understandand show them examples of excellent work, the likelihood of students attaining the criteria is greatly increased (McTighe & Ferrara, 1998; OMalley & Pierce, 1996; Stiggins, 2002). The second key element of performance-based assessment is self-assessment, which is essential for teaching students how to manage their study habits, use learning strategies, and reflect on progress toward learning goals. The goal of self-assessment is to produce students who can learn independently of the teacher and become lifelong learners. To accomplish this, teachers need to provide students with specific feedback, opportunities to give and receive

ERIC/CLL n e w s bulletin fall 2002 page 2

feedback from peers, and time to set learning goals. Self-assessment also plays a role in motivating learners to continue learning and building self-confidence in their ability to learn. Performance-based assessments that are designed for the language proficiency level of ELLs, that call for transfer of learning through meaningful tasks, that make criteria for evaluation visible to students, and that show ELLs how to monitor their own work can also support learning for these students.

assessment focus groups can lead assessment changes in each school system. Teachers need to find their voices and become active in shaping their own professional development in order to improve learning for ELLs.

Conclusion
Teachers of ELLs work in school environments that are increasingly under pressure to prepare these students to pass standardized tests for accountability purposes. Closing the achievement gap between language minority and non-minority students will also require improved assessments that research shows can promote and support learning at the classroom level. While standardized tests may be appropriate for determining whether or not students have met state and local standards, we need other forms of assessment to inform instructional decisions made on a day-to-day basis, diagnose students strengths and weaknesses related to classroom instruction, and provide specific feedback to students that supports their learning. For this purpose, we need classroom-based assessments that reflect instructional activities and learning standards, make clear achievement targets, and help teachers redirect instruction to promote learning. Although not a panacea, performancebased assessments can promote increased achievement for ELLs by increasing confidence in their ability to learn and motivation to continue learning. If we are going to be successful in closing the achievement gap, we will need to find new ways to support student learning and make improved assessment practices available to teachers of ELLs. These children deserve no less.

Becoming Assessment Literate

Most assessments used in the classroom are developed by teachers, and these assessments of student work have more influence on instructional decisions than state-mandated tests (Frisby, 2001; Wiggins, 1998). Yet very few teachers have access to the type of assessment information that will enable them to assess ELLs accurately and fairly. Indeed, the vast majority of teachers report that they feel unprepared to assess and teach ELLs (Fradd & Lee, 2001). Only about a dozen states require teacher candidates to show competence in assessment in order to get a teaching license, and the majority of teacher preparation programs fail to provide instruction in developing assessments that support student learning (Herman, Aschbacher, & Winters, 1992; Stiggins, 2002). Most teachers use the same types of tests that were used when they were in school, typically traditional multiplechoice, fill-in-the-blank, matching, and true/false tests. In fact, little has changed in classroom-based assessment for at least the past 50 years (Bertrand, 1994). To be able to use improved classroombased assessment practices such as performance-based assessments, teachers must have access to professional development opportunities that will help them learn to design and use assessments that can improve the achievement of ELLs. Mere exposure to learning or assessment theories or examples of innovative assessments will not be enough, and neither will one-shot workshops. The kind of professional development that is needed is of a longterm, collaborative nature that helps teachers try out their assessments and get feedback from colleagues, program administrators, and university faculty experienced in using assessments for learning. School study teams and

References
Amrein, A. L., & Berliner, D. C. (2002). Highstakes testing, uncertainty, and student learning. Education Policy Analysis Archives, 10 (18). Retrieved May 29, 2002, from http:// epaa.asu.edu/epaa/v10n18/. Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80 (2), 141-148. Bertrand, J. E. (1994). Student assessment and evaluation. In B. Harp (Ed.), Assessment and evaluation for student centered learning (pp. 2745). Norwood, MA: Christopher-Gordon. Calkins, L., Montgomery, K., & Santman, D. (1998). A teachers guide to standardized reading tests. Portsmouth, NH: Heinemann. Darling-Hammond, L., Ancess, J., & Falk, B. (1995). Authentic assessment in action: Studies

of schools and students at work. New York: Teachers College Press. Feuer, M. J., & Fulton, K. (1993). The many faces of performance assessment. Phi Delta Kappan, 74 (6), 478. Fradd, S. H., & Lee, O. (2001). Needed: A framework for integrating standardized and informal assessment for students developing academic language proficiency in English. In S. R. Hurley & J. V. Tinajero (Eds.), Literacy assessment of second language learners (pp. 130148). Boston: Allyn & Bacon. Frisby, C. L. (2001) Academic achievement. In L. A. Suzuki, J. G. Ponterotto, & P. J. Meller (Eds.), Handbook of multicultural assessment (2nd ed., pp. 541-568). San Francisco: JosseyBass. Gardner, H. (1999). The disciplined mind: What all students should understand. New York: Simon & Schuster. Hamayan, E. V., & Damico, J. S. (1991). Limiting bias in the assessment of bilingual students. Austin, TX: Pro-Ed. Herman, J. L., Aschbacher, P. R., & Winters, L. (1992). A practical guide to alternative assessment. Alexandria, VA: Association for Supervision and Curriculum Development. Heubert, J. P., & Hauser, R. M. (Eds.). (1999). High-stakes: Testing for tracking, promotion, and graduation. Washington, DC: National Academy Press. Kohn, A. (2000). The case against standardized testing. Portsmouth, NH: Heinemann. Kolls, M. (1992, March). Portfolio assessment: A feasibility study. Paper presented at the annual meeting of Teachers of English to Speakers of Other Languages, Vancouver, B.C., Canada. Marzano, R. J., Pickering, D., & McTighe, J. (1993). Assessing student outcomes: Performance assessment using the Dimensions of Learning model. Alexandria, VA: Association for Supervision and Curriculum Development. McKeon, D. (May 1994). When meeting common standards is uncommonly difficult. Educational Leadership, 51 (8), 45-49. McTighe, J., & Ferrara, S. (1998). Assessing learning in the classroom. Washington, DC: National Education Association. No Child Left Behind Act (2001). Part A, Improving Basic Programs operated by Local Education Agencies, Subpart 1, Basic Program Requirements. Section 1001 (3): Statement of purpose. OMalley, J. M., & Pierce, L. V. (1996). Authentic assessment for English language learners: Practical approaches for teachers. New York: Longman. Stiggins, R. J. (2002). Assessment crisis: The absence of assessment FOR learning. Phi Delta Kappan, 83 (10), 758-765. Wiggins, G. (1998). Educative assessment: Designing assessments to inform and improve student performance. San Francisco: Jossey-Bass.

ERIC/CLL ne w s bulletin fall 2002 page 3

PERFORMANCE-BASED ASSESSMENT AND THE EFL CLASSROOM: COMPILATION Implementing Performance Assessment in the Classroom Amy Brualdi, ERIC/AE Brualdi, A. (1998). Implementing performance assessment in the classroom. Practical Assessment, Research & Evaluation, 6(2). Retrieved from http://pareonline.net/getvn.asp?v=6&n=2 . Introduction If you are like most teachers, it probably is a common practice for you to devise some sort of test to determine whether a previously taught concept has been learned before introducing something new to your students. Probably, this will be either a completion or multiple choice test. However, it is difficult to write completion or multiple choice tests that go beyond the recall level. For example, the results of an English test may indicate that a student knows each story has a beginning, a middle, and an end. However, these results do not guarantee that a student will write a story with a clear beginning, middle, and end. Because of this, educators have advocated the use of performance-based assessments. Performance-based assessments "represent a set of strategies for the . . . application of knowledge, skills, and work habits through the performance of tasks that are meaningful and engaging to students" (Hibbard and others, 1996, p. 5). This type of assessment provides teachers with information about how a child understands and applies knowledge. Also, teachers can integrate performance-based assessments into the instructional process to provide additional learning experiences for students. The benefits of performance-based assessments are well documented. However, some teachers are hesitant to implement them in their classrooms. Commonly, this is because these teachers feel they don't know enough about how to fairly assess a student's performance (Airasian,1991). Another reason for reluctance in using performance-based assessments may be previous experiences with them when the execution was unsuccessful or the results were inconclusive (Stiggins, 1994). The purpose of this digest is to outline the basic steps that you can take to plan and execute effective performance-based assessments. Defining the Purpose of the Performance-Based Assessment In order to administer any good assessment, you must have a clearly defined purpose. Thus, you must ask yourself several important questions:

What concept, skill, or knowledge am I trying to assess? What should my students know? At what level should my students be performing? What type of knowledge is being assessed: reasoning, memory, or process (Stiggins, 1994)?

By answering these questions, you can decide what type of activity best suits you assessment needs.

Choosing the Activity After you define the purpose of the assessment, you can make decisions concerning the activity. There are some things that you must take into account before you choose the activity: time constraints, availability of resources in the classroom, and how much data is necessary in order to make an informed decision about the quality of a student's performance (This consideration is frequently referred to as sampling.). The literature distinguishes between two types of performance-based assessment activities that you can implement in your classroom: informal and formal (Airasian, 1991; Popham, 1995; Stiggins, 1994). When a student is being informally assessed, the student does not know that the assessment is taking place. As a teacher, you probably use informal performance assessments all the time. One example of something that you may assess in this manner is how children interact with other children (Stiggins, 1994). You also may use informal assessment to assess a student's typical behavior or work habits. A student who is being formally assessed knows that you are evaluating him/her. When a student's performance is formally assessed, you may either have the student perform a task or complete a project. You can either observe the student as he/she performs specific tasks or evaluate the quality of finished products. You must beware that not all hands-on activities can be used as performance-based assessments (Wiggins, 1993). Performance-based assessments require individuals to apply their knowledge and skills in context, not merely completing a task on cue. Defining the Criteria After you have determined the activity as well as what tasks will be included in the activity, you need to define which elements of the project/task you shall to determine the success of the student's performance. Sometimes, you may be able to find these criteria in local and state curriculums or other published documents (Airasian, 1991). Although these resources may prove to be very useful to you, please note that some lists of criteria may include too many skills or concepts or may not fit your needs exactly. With this in mind, you must be certain to review criteria lists before applying any of them to your performance-based assessment. You must develop your own criteria most of the time. When you need to do this, Airasian (1991, p. 244) suggests that you complete the following steps: 1. Identify the overall performance or task to be assessed, and perform it yourself or imagine yourself performing it 2. List the important aspects of the performance or product. 3. Try to limit the number of performance criteria, so they can all be observed during a pupil's performance. 4. If possible, have groups of teachers think through the important behaviors included in a task. 5. Express the performance criteria in terms of observable pupil behaviors or product characteristics. 6. Don't use ambiguous words that cloud the meaning of the performance criteria.

7. Arrange the performance criteria in the order in which they are likely to be observed. You may even wish to allow your students to participate in this process. You can do this by asking the students to name the elements of the project/task that they would use to determine how successfully it was completed (Stix, 1997). Having clearly defined criteria will make it easier for you to remain objective during the assessment. The reason for this is the fact that you will know exactly which skills and/or concepts that you are supposed to be assessing. If your students were not already involved in the process of determining the criteria, you will usually want to share them with your students. This will help students know exactly what is expected of them. Creating Performance Rubrics As opposed to most traditional forms of testing, performance-based assessments don't have clearcut right or wrong answers. Rather, there are degrees to which a person is successful or unsuccessful. Thus, you need to evaluate the performance in a way that will allow you take those varying degrees into consideration. This can be accomplished by creating rubrics. A rubric is a rating system by which teachers can determine at what level of proficiency a student is able to perform a task or display knowledge of a concept. With rubrics, you can define the different levels of proficiency for each criterion. Like the process of developing criteria, you can either utilize previously developed rubrics or create your own. When using any type of rubric, you need to be certain that the rubrics are fair and simple. Also, the performance at each level must be clearly defined and accurately reflect its corresponding criterion (or subcategory) (Airasian, 1991; Popham, 1995; Stiggins, 1994). When deciding how to communicate the varying levels of proficiency, you may wish to use impartial words instead of numerical or letter grades (Stix, 1997). For instance, you may want to use the following scale: word, sentence, page, chapter, and book. However, words such as "novice," "apprentice," "proficient and "excellent" are frequently used. As with criteria development, allowing your students to assist in the creation of rubrics may be a good learning experience for them. You can engage students in this process by showing them examples of the same task performed/project completed at different levels and discuss to what degree the different elements of the criteria were displayed. However, if your students do not help to create the different rubrics, you will probably want to share those rubrics with your students before they complete the task or project. Assessing the Performance Using this information, you can give feedback on a student's performance either in the form of a narrative report or a grade. There are several different ways to record the results of performancebased assessments (Airasian,1991; Stiggins,1994):

Checklist Approach When you use this, you only have to indicate whether or not certain elements are present in the performances. Narrative/Anecdotal Approach When teachers use this, they will write narrative reports of

what was done during each of the performances. From these reports, teachers can determine how well their students met their standards. Rating Scale Approach When teachers use this, they indicate to what degree the standards were met. Usually, teachers will use a numerical scale. For instance, one teacher may rate each criterion on a scale of one to five with one meaning "skill barely present" and five meaning "skill extremely well executed." Memory Approach When teachers use this, they observe the students performing the tasks without taking any notes. They use the information from their memory to determine whether or not the students were successful. (Please note that this approach is not recommended.)

While it is a standard procedure for teachers to assess students' performances, teachers may wish to allow students to assess them themselves. Permitting students to do this provides them with the opportunity to reflect upon the quality of their work and learn from their successes and failures. References and Additional Reading Airasian, P.W. (1991). Classroom assessment. New York: McGraw-Hill. Hibbard, K. M. and others. (1996). A teacher's guide to performance-based learning and assessment. Alexandria, VA: Association for Supervision and Curriculum Development. Popham, W. J. (1995). Classroom assessment: What teachers need to know. Needham Heights, MA: Allyn and Bacon. Stiggins, R. J. (1994). Student-centered classroom assessment. New York: Macmillan Publishing Company. Stix, A. (1997). Empowering students through negotiable contracting. (Paper presented at the National Middle School Initiative Conference (Long Island, NY, January 25, 1997) (ERIC Document Reproduction Number ED411274) Wiggins, G. (1989). A true test: Toward more authentic and equitable assessment. Phi Delta Kappan, May, 703-713. Wiggins, G. (1993). Assessment, authenticity, context, and validity. Phi Delta Kappan, November, 200-214. Wiggins,G. (1998). Educative assessment: designing assessments to inform and improve student performance San Francisco, Calif.: Jossey-Bass.

EXAMPLES OF RATING SCALES Nakamura, Y. (2004). A comparison of holistic and analytic scoring methods in the assessment of writing . Paper presented at the 3rd Annual JALT Pan-SIG Conference: The Interface Between Interlanguage, Pragmatics and Assessment, Tokyo, Japan. Retrieved from http://jalt.org/pansig/2004/HTML/Nakamura.htm Holistic rating scale: A four-point scale (1, 2, 3, 4) 4 points the main idea was clearly stated the essay was well organized the choice of words was good very few minor grammatical errors 3 points the main idea was fairly clear the essay was moderately well organized the vocabulary was good some minor grammatical errors 2 points the main idea was indicated, but not clearly the essay was not so well organized the vocabulary choice was fair some major grammatical errors 1 point the main idea was hard to identify the essay was poorly organized the vocabulary was weak many grammatical errors

Analytic rating scale: A four-point scale (1, 2, 3, 4) Originality of Content

4 points: inter esting ideas were stated clearly 3 points: inter esting ideas were stated fairly clearly 2 points: idea s somewhat unclear 1 point: ideas not clear

Organization
4 points: well organized 3 points: fairl y well organized 2 points: loos ely organized 1 point: ideas disconnecte d

Vocabulary
4 points: very effective choice of words 3 points: effe ctive choice of words 2 points: fairl y good vocabulary 1 point: limite d vocabulary range of vocabulary

Grammar
4 points: alm ost no errors 3 points: few minor errors 2 points: som e errors 1 point: many errors

Cohesion & Logical Consistency

4 points: sent ences logically combined 3 points: sent ences fairly logically combined 2 points: sent ences poorly combined 1 point: many unfinished sentences