Who barged into my testing?

Apresentação em tema: "Who barged into my testing?"— Transcrição da apresentação:

1 Who barged into my testing?
Miriam Sester Retorta

2 The Scream (Edvard Munch)

3

4 Think about the mistakes that your students make, or the mistakes you make yourself when using a language that is not your mother tongue. How do you feel about them? Please, write three sentences using the word “Mistake” in them!!!!

5 Think Globally, Act Locally No teaching is effective if it is not affective

6 What is evaluation? What are the reasons for anxiety and the derogatory image of tests? Common questions teachers ask: * Should we evaluate the student with numerical grades, alphabet grades, descriptive reports? * How can we develop effective evaluation instruments? * How can we establish fair criteria to evaluate someone? * When should we flunk a student? * What criteria should we use to evaluate written work?

7 Common beliefs and uses:
It is an action of judgment of results achieved, analysis of performance, measurement of capacity, evaluate ‘the whole of the student’. Selective and discriminatory function of grades and concepts: serious damage due to failure. Bureaucratic, punitive functions which hinder life projects of many people.

8 What it should be: Evaluation is essential to education. It is inherent and indissoluble when it is related to questioning, inquiring, reflecting about our students’ and our own actions. Problem: educators generally view teaching and evaluating as two distinct moments and not interrelated !!!! ‘Mistakes are seen as failure and doubts as lack of knowledge’.

9 Jussara Hoffmann, (2001) In an educational dimension students’ mistakes and doubts are considered a highly significant episode which lead to educational actions. In this dimension evaluate is to create the opportunity to reflect about the actions taken in the classroom in a permanent teacher watch, which will instigate students to new questions from previously answered questions”. Coherent evaluative practice: 1) It demands from teachers deep and detailed studies of theories of knowledge. 2) It demands a vision, at the same time, broad and detailed of his/her discipline. 3) It demands an eternal search of comprehension of students’ difficulties and the search of new opportunities to develop knowledge. 4) We need to know like our students do and find out why they think that way.

10 Exemple: doctor-patient and tests.
Why evaluate We cannot reduce an instrument of evaluation to results only: to select or classify. We must use it to improve our educational actions. Evaluate is an investigative procedure so we can go way further in the accompanying the process of knowledge construction. Evaluate is an instrument of investigation of BOTH students and teacher. (sometimes even school and curriculum) DON’Ts: The only notes a teacher writes on the test are: super, great, fair, poor, study harder, pay attention, incomplete answer, etc – Generic recommendations !!! Exemple: doctor-patient and tests. It is not enough to diagnose the disease. You must offer adequate treatment !!!!! Doctor, engineer and teacher – success or failure Assignments: group work (trabalho em grupo ou equipe), individual paper, compositions– CRITIRION OF CORRECTION. ERROR ANALYSIS - to diagnose what is the real problem. Must be done WITH the student !!!

11 If making mistakes is part of learning, and correction is part of teaching, how do the two of them go together????

12 Mistakes The most important sort of mistakes is a mistake that leads to a misunderstanding. (examples from: Mistakes and Correction. Julian Edge, Longman)

13 Mistake 1 Imagine a student, Cheng, who wants to buy a jacket in Britain. She goes into a shop and says: Please will you to show me coats? (What is the most important mistake here?)

14 Mistakes 2 Peter: How long are you here for? Amira: Two years
Peter: Wha...! You are already here since two years? Amira: No, no. I am come yesterday. Peter: Oh, yester.... Amira: No, no, last week. I mean I came last week. (What is the most important mistake here?)

15 Mistake 3 Nesrin: Good Morning. Shopkeeper: What can I do for you?
Nesrin: Give me two apples. (Shopkeeper weighs two apples, puts them in a bag and gives them to Nesrin) Shopkeeper:Anything else? Nesrin: No. How much this? Shopkeeper: For pence.

16 People don’t usually mind if foreigners make grammatical mistakes....
But people are not very forgiving of social mistakes. It is more important to correct mistakes which affect the meaning of several sentences than to correct small grammatical points inside one sentence!!! A n Italian who went to New York.

17 Exemplo 1: Juliana 2ª série ensino fundamental – prova de matemática.
* Leonora tem 15 balas. Leonel tem 8 balas. Quantas balas Leonora tem a mais que Leonel? Juliana: Leonora tem 7 balas a mais que Leonel. 8+7=15 Princípio da reversibilidade: capacidade de reverter, relações, realizar operações inversas. Exemplo 2: relato de uma estagiária “Ao visitar uma sala de crianças maiores e analisar a decoração, verifiquei a existência de trabalhos das crianças: eram flores de papel plantadas em vasos com areia. Inicialmente fiquei feliz ao notar a presença de algum trabalho delas. Porém, logo chamou-me a atenção o fato de os vasos serem idênticos e, mais do que isso, as flores serem todas iguais. “Encucada” questionei as crianças acerca do trabalho. Desse questionamento retirei que quem havia feito os vasos fora a professora e que também ela é que havia colocado areia dentro, também a professora desenhou as pétalas das flores, também a professora colocou o cabinho (todas da mesma altura) e também foi ela que colou as pétalas. Quando perguntei: -Tu colaste estas pétalas tão bonitas? Criança: Não, porque eu tinha colado todas assim juntos, então a professora arrumou. A essa altura cabe a pergunta: Mas o que enfim as crianças fizeram? Qual a concepção de avaliação que parece evidenciar a professora observada pela estagiária? (Quantas vezes não se responde ao aluno sem dar tempo sequer dele pensar? A professora quis garantir o resultado esperado. )

18 Exemplo 3: Marcelo é aluno da 3ª série do ensino fundamental
Exemplo 3: Marcelo é aluno da 3ª série do ensino fundamental. Prova de matemática Numere as linhas do caderno de matemática de 1 a 10. Escrevam apenas os resultados das seguintes multiplicações: (e ditou) 3x7; 4x7; 8x7; 9x7; 2x7; 6x7; 5x7; 1x7; 7x7 e 10x7. Marcelo respondeu no caderno: Após o exercício, os alunos trocaram de cadernos e corrigiram os exercícios uns dos outros. O colega de Marcelo atribuiu-lhe 2 pontos. A professora revisou a correção e confirmou o resultado. Como você, professor(a), procederia no lugar da professora de Marcelo?? © Professora : ele memorizou a tabuada na seqüência Pesquisador descobriu: O menino relacionou aos itens que a professora mandou enumerar. © Exemplo 4: Uma criança da 4ª série. Estuda o tema “características dos mamíferos” para prova bimestral. (no livro)* A temperatura dos mamíferos é constante. * suas garras e dentes são adaptados ao meio ambiente.

19 A professora acompanha a leitura da menina e pergunta:
O que é ‘temperatura’? - O que é constante? O que é meio ambiente dos animais? O que quer dizer adaptados? Exemplo 5: A professora perguntou: - O que é desmatar? A aluna: - tornar vivo de novo!!!! Depois de analisar o problema a professora chega a conclusão: vocabulário do dia-a-dia: desarrumar, despentear. Perguntas: O meu aluno compreende? O meu aluno Não compreende? Por quê? Exemplos adaptados do livro: Avaliação: Mitos e Desafios – Jussara Hoffmann

20 Slips, errors and attempts
Slip: If the teacher thinks that a student could self-correct a mistake. Eg1 Error: If a student cannot self- correct a mistake in his or her own English, but the teacher thinks that the class is familiar with the correct form. EG2 Attempt: When the teacher knows that the students have not yet learned the language necessary to express what they want to say.Eg3 How would you grade/evaluate your students??????????

21 Be careful! * I wish I went my grandmother’s house last summer.
This could be a slip for one student, an error for another, and an attempt for a third. Or for the same student: it might be an attempt today, an error next month and a slip next year....

22 Learning Steps A child is learning English as its mother tongue.
Nina catched a balloon. Nina got two foots. A student (learning English as a FL) in English 2: I goed to school yesterday. (In time, this child/ student will learn that these two very good and useful rules also have exceptions to them. These examples are clear signals that the learning steps are being made.)

23 Learning Steps But many of the things we call mistakes and see as problems are in fact signals that our students are successfully learning the language. They are taking the necessary learning steps. Now substitute the word “Mistake” in your sentences to “Learing Steps” and see what you’ve got!!!!

24 Teacher’s Role Our job as teachers is NOT just to point out differences between our students’ language and standard English. That is too negative a role. Our job is also to encourage the growth of the language by appreciating the learning steps. (And show this process to the students!!!)

25 When should we correct students???
Risks and chances Fluency and accuracy

26 History of testing The Pre-scientific Era (before the 50’s) –(Spolsky) or Garden of Eden (Morrow) The Psychometric-Structuralist Era (from 50’s to end of 60’s) –(Spolsky) or The Vale of Tears (Morrow) The Psycholinguistic-Sociolinguistic Era (end of 60’s up to our days) –(Spolsky) ou The Promised Land (Morrow)

27 The Pre-scientific Era (Spolsky) or Garden of Eden (Morrow)
It was not based on scientific knowledge, and for this reason, the idea of testing a language as a distinct activity did not exist. According to Spolsky, developing a test was art. Theoretical basis did not existed and the methods were made up from impressionist observations of their creators. They used the same activities from the classroom to evaluate progress: translation, dictation and composition. There was no clear difference between testing and teaching. Garden of Eden – where there were no worries and there were no distinctions of good and evil. There were no questioning. It lasted from the beginning of language teaching to Second World War.

28 Grammar-translation Test
1. Dê a correta forma do presente do indicativo do verbo “to be”. He _______ a good student. They ________ old friends It _______ on the other side of town. We _______ cousins. Mr.Smith _______ out of town. 2. Traduza para o português as orações abaixo: They are good students. _____________________________ His wife is sick. ________________________. We are busy today. __________________________ She is his cousin. __________________________ 3. Agora, faça a versão para o inglês. Ele é um bom amigo meu. _________________________________. Ele está enfermo. _____________________________________. Não há aulas no Domingo. ______________________________. O João está ausente na aula hoje. ____________________________

29 Reading test (30’s and 60’s)
A Famous Composer One day, a young man went to visit a famous composer and asked him for advice on how to compose a symphony. “How old are you?” the composer asked. “Eighteen.” “I think that you are too young to compose a symphony,” then the composer told him. “Try something easier.” “But you composed symphonies when you were eight,” protested the young man. “Yes,” agreed the famous composer, who was Mozart, “but I did not ask anyone how to compose them.” (adapted from “English I” p15 – Amadeu Marques, 1978, Ed Ática) Refer to the reading and write or complete the answers to the following questions: Who was the famous composer? Why did the young man visit the famous composer? What did Mozart ask the young man? How old was the young man? Did Mozart advise the young man to compose another symphony? What did Mozart advise the young man? Was the young man too old to compose a symphony? Did the young man protest? What did Mozart do when he was eight? Did Mozart ask anyone how to compose a symphony?

30 Marcuschi, L.A. Exercícios de compreensão ou copiação nos manuais de ensino de língua? In: Em Aberto, Brasília, ano 16, n.69, jan/mar, 1996 Tipologia de questões abertas de interpretação de textos: “Falta de Horizonte” – apenas há repetição ou cópias do que está dito no texto. “Horizonte Mínimo” – paráfrases são utilizadas, ou seja, uma espécie de repetição com outras palavras. “Horizonte Máximo” – inferências: atividades de geração de sentidos pela reunião de várias informações do próprio texto, ou pela introdução de informações e conhecimentos pessoais outros não contidos no texto. “Horizonte Problemático” – São perguntas que vão muito além das informações do próprio texto. São leituras de caráter idiossincrático, bem pessoal, onde o investimento de conhecimentos pessoais é muito grande e chega a ser preocupante. “Horizonte Indevido” – leitura errada. Nery, M. Rosa Questões sobre questões de leitura. Campinas, SP. Alínea Editora, 2003.

31 The Psychometric-Structuralist Era (1920’s-60’s) Spolsky The Vale of Tears (Morrow)
Baker (1989:29) The 1920s and 30s saw a great vogue for psychological testing. Large numbers of tests investigating every aspect of the psyche from intelligence to job aptitudes were produced and millennial predictions were sometimes made about the social benefits that large-scale testing of this kind would bring. Few of these tests actually delivered the miraculous solutions which had been promised but they survive today in the form of intelligence tests and, in a less serious form, as magazines quizzes of the kind “Are you a good husband?” 1. Behaviorist psychology offered us concepts such as : the multiple-choice method – descrete-pont tests (testes de itens isolados); terms such as reliability, validity, practicality, psychometric

32 The Psychometric-Structuralist Era (1920’s-60’s) Spolsky The Vale of Tears (Morrow)
2. According to the academic tradition which helped define the limits of language testing –Structuralist linguistics. They analyzed the fragmented linguistic system in bits of pieces and then described possible ways the bits and pieces could be united again to form bigger pieces of speech. Hierarchy – phonemes, words, phrases, etc. “Language was a huge puzzle”.

33 The Psychometric-Structuralist Era (1920’s-60’s)
Vestibular UFPR (1977) Descrete-point tests = Questões de itens isolados 48. Assinale a alternativa correta. Mr. & Mrs.Brown invited us to ____ house, and there we met a cousin of ______. their, his Our, her Their, theirs His, his Her, hers 49. The girls ______ were here this morning are probably Sally´s friends. a) Which b) whom c) who d) whose e) who´s

34 Hymes (1967, 1972), Canale&Swain(1980), Canale(1982)
The Psycholinguistic-Sociolinguistic Era (end of 60’s up to our days) –(Spolsky) ou The Promised Land (Morrow) Psycholinguistics – Frank Smith (1978a/b); Kenneth Goodman (1967, 1976a/b) Hymes (1967, 1972), Canale&Swain(1980), Canale(1982)

35 The Psycholinguistic-Sociolinguistic Era (end of 60’s up to our days) –(Spolsky) ou The Promised Land (Morrow) 70’s/80’s? From a psycholinguistic perspective, language was seen less as a taxonomic structure well defined and more like a creative and dynamic functional system The contribution of sociolinguistic was centered on the conception of communicative competence which not only covered the knowledge of rules to form grammatical sentences, but also on rules of adequate usage of sentences in different contexts. Therefore the psycholinguistic and sociolinguistic perspective helped develop basis for valid testing.

36 The Psycholinguistic-Sociolinguistic Era
Dictation. Prepare to take dictation. ________________________________________________________________________________________________________________________________________________________________________________________________________________________ 2. Fill int the gaps in the passage below with na appropriate word. The history of money is very interesting. Today money is used everywhere to buy goods and services. In the past _________were among the earliest objects used as money. These were originally used as _________ but gradually the people in the Pacific began to use them in exchange for food and other items. In other countries many different objects were used as money. Rice was used in the ___________, whereas in Africa they used ________ which is still in use as money in certain parts of that continent today. A person´s weath was often demonstrated by the number of _________ he owned and it was even possible in some countries to buy a ___________ with these animals Coins came in many shapes and sizes. In Russia they were made from ________ but in _______ they were made from leather. The _________ of the coins, however, was the main factor which led to the invention of money made from __________. 3. Compositon: Write about what you did on your last vacation!!!! __________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

37 Where are we heading to?

38 Communicative paradigm of testing
Objectives: 1. Try to simulate tasks that learner will perform in the future. Integrate abilities Authentic and contextualized content Qualitative and holistic corrections Direct vs indirect Performance vs knowledge Exemple: Celpe-Bras Certificado de Língua Portuguesa para Estrangeiros – Ministério da Educação.

39 Types of testing Proficiency tests Achievement Diagnostic Placement
Entrance Mastery (esp)

40 Concepts Validity - construct - face - content reliability
Practicality Always in tension validity reliability practicality

41 Validity, reliability and practicality
Content – does your test consist of a respresentative sample of the abilities and sub-abilities worked in class? Construct – when a test evaluates an ability (not only the content) that it is intended to evaluate. Face - Does your test look like a reliable test? Reliability It answers the following questions: Will the same evaluator grade the same tests, the same way in different times? Will different evaluators grade the same at the same time? Did the test allow the testtakers to grade the same at different times? Practicality - A test must be feasible within a limit of time and budget. It must have high cost-benefit.

42 What are students’ views of language tests?
St1 – We spent ten lessons conjugating the past tense but on the test there were only two conjugations. St2 – I never learn anything from tests because the teacher never corrects the mistakes I make, so I end up at the same place where I was before I took the test, except now I also have a bad grade. St3 – The teacher bases the grade on one test, so if I do not feel well on the same day that the test was given, I flunk the course. St4 – I don’t believe that any test is a good measure of my performance, let alone on a day when I do not feel well. St5 – I don’t see the connection between the test and my knowledge, otherwise, how can I explain the fact that I get good grades on English tests, but last week, when I met a n American, I couldn’t say anything in English? How come we never speak on tests?

43 What are students’ views of language tests?
St6 – Our teacher uses the test as a punishment, whenever we don’t behave she tells us we will have a test the following day. Sometimes she even asks us to take out a sheet of paper and write the test on the spot. St7- A test really makes me study, I would have never opened a book if not the test. I think the pressure is good for me. St8 – I hate to flunk, tests always show me that I am a failure. St9 – It seems that whenever the teacher is unprepared she asks us to write a test. St10 – I don’t mind the test, what I do mind is that it takes the teacher such a long time to correct the tests. By the time I get the test back, I forgot what the test was all about.

44 What are students’ views of language tests?
St11 – I think that it is really strange that whenever I study hard, I don’t get a good grade but when I don’t study at all, I happen to succeed. Boes it say something about me or about the test? St12 – What I hate the most is when the teacher does not tell us in advance what the test will cover. It seems that I am always studying the wrong things. St13 – Why do we need tests? The teacher knows how well we are doing anyway.

45 What’s wrong with my test?
Using tests as a disciplinary tool – as a punishment. There is a gap between what is taught and what is tested. There is a delay in returning test papers so the feedback, when given, becomes irrelevant. There is a n absence of stisfactory corrections and explanations of erros – too little information that could be used by students for correction/improvement. There is a feeling of failure that results from obtaining poor results on test.

46 What’s wrong with my test?
Teachers themselves do not feel confident in their tests. There is little knowledge on the part of the teachers as to the purpose and uses of language tests and their role in language learning. Using tests exclusively for grading. Using tests without propor planning. No coorporation among teachers about tests; no analysis of tests after they have been administered.

47 What’s wrong with my test?
Not discarding bad tests or bad items Not preparing students for tests or for testing procedure. Not giving feedback about tests Lack of testing knowledge and training Using one testing method to tap given knowledge. Using tests which were intended for different populations and not checking if they are appropriate for a given population.

48 The Multiple Intelligences
by Howard Gardner - Frames of Mind: The Theory of Multiple Intelligences * Gardner suggested there were at least 7 different kinds of intelligences, i.e., 7 different ways of perceiving and “knowing” the world . This also suggested that people solve problems which imerge in every day life using 7 different learing styles.

49 a

50 Logical / Number smart - You may be number smart
Logical / Number smart - You may be number smart. You will be good at mathematics and other number activities; you are also good at solving problems. This is sometimes called being Logical smart.  

51 a

52 Linguistic/ Work smart You may be word smart
Linguistic/ Work smart You may be word smart. You will enjoy reading, writing and talking about things. This is sometimes called being Linguistic smart.

53 a

54 Kinaesthetic / Body smart – You may be body smart
Kinaesthetic / Body smart – You may be body smart. You will enjoy sports and are good at swimming, athletics, gymnastics and other sports. This is sometimes called being Kinaesthetic smart.

55 a

56 Musical – Music Smart You may be music smart
Musical – Music Smart You may be music smart. You will enjoy music and can recognise sounds, and timbre, or the quality of a tone. This is sometimes called being Musical smart.

57 a

58 Visual/Spatial-Picture smart You may be picture smart
Visual/Spatial-Picture smart You may be picture smart. You will be good at art and also good at other activities where you look at pictures like map reading, finding your way out of mazes and graphs. This is sometimes called being Visual/Spatial smart.

59 a

60 Interpersonal /People smart You may be people smart
Interpersonal /People smart You may be people smart. You will like to mix with other people and you will belong to lots of clubs. You like team games and are good at sharing. This is sometimes called being Interpersonal smart.  

61 a

62 Intrapersonal / Myself smart You may be myself smart
Intrapersonal / Myself smart You may be myself smart. You will know about yourself and your strengths and weaknesses. You will probably keep a diary. This is sometimes called being Intrapersonal smart.  

63 Did anyone barge into your testing ?????