28 August 2012Ganesha Associates8 Source: http://www.scimagoir.com/
Global research investment 28 August 2012Ganesha Associates9
28 August 2012Ganesha Associates10 The need for assessment in science “Afinal das contas é tudo uma questão de transformar conhecimento em riqueza e bem- estar para a população.” “When I was in graduate school I thought I would go into a room and people would slide money through a slot in the door, I would do science, and then slide the results back out. But as my career has gone on, they are increasingly opening the door and asking me what I am doing with their money.”
28 August 2012Ganesha Associates11 O Cientista Brasileiro é avaliado ?! –À primeira vista, o papel do cientista parece ser tão-somente expandir as fronteiras do conhecimento humano. Porém, o contrato social da ciência prevê que a informação gerada pelo cientista deva ter por fim modificar a realidade para a qual ele produziu a informação. Mas isso nem sempre ocorre. E a avaliação é um processo que possibilita à sociedade e aos gestores de pesquisa meios de aferir a real efetividade do trabalho do cientista. –O cientista, também, está sempre solicitando mais verbas ao governo para publicar cada vez mais informação. E a avaliação, novamente, é uma importante ferramenta para que se concedam verbas aos mais produtivos e inventivos. –Assim, a avaliação do cientista deve considerar o cotidiano do pesquisador, respeitando as demandas da sociedade, que nem sempre são apenas publicar artigos e livros que só interessarão a outros cientistas. –Atualmente, o desempenho dos cientistas é avaliado através das citações por eles obtidas, por os artigos publicados só nas revistas indexadas no Science Citation Index (SCI), ou Medline ou Scielo. –Taken from Pedrini, Alexandre de Gusmão, 2005
Levels of quality World-leading in terms of originality, significance and rigour. Internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence. Recognised internationally in terms of originality, significance and rigour Recognised nationally in terms of originality, significance and rigour. Falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment. 5 July 2012Copyright: Ganesha Associates 2012 12
5 July 2012Copyright: Ganesha Associates 2012 13
28 August 2012Ganesha Associates14 Como o/a cientista é avaliado ?! –National programs for scientific research have targets –National wealth and well-being –‘Basic science with a purpose’ –Getting funding is a highly competitive process –The scientist has an obligation to deliver results –We need management tools in science. –The quality of scientific output is one measure –Methods of scientific assessment include the analysis of citation statistics
28 August 2012Ganesha Associates15 Qualis e a classificação de periódicos, anais, revistas e journais Qualis é o resultado do processo de classificação dos veículos utilizados pelos programas de pós-graduação para a divulgação da produção intelectual de seus docentes e alunos Tal processo foi concebido pela CAPES para atender as necessidades específicas do sistema de avaliação e baseia-se nas informações fornecidas pelos programas e pelo Coleta de Dados A classificação é feita ou coordenada pelo representante de cada área e passa por processo anual de atualização. Os veículos de divulgação citados pelos programas de pós- graduação são enquadrados em categorias indicativas da qualidade - A, B ou C e do âmbito de circulação dos mesmos - local, nacional ou internacional As combinações dessas categorias compõem nove alternativas indicativas da importância do veículo utilizado, e, por inferência, do próprio trabalho divulgado.
Avaliação e o conceito CAPES Os Programas de Pós-Graduação da Universidade Federal de Pernambuco ocupam posições de destaque na avaliação elaborada pela Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – CAPES. No total, são oferecidos na UFPE 65 cursos de mestrado acadêmico, seis cursos de mestrado profissional e 45 cursos de doutorado. Um terço desses cursos recebeu os conceitos 5 e 6 – uns dos mais altos da Avaliação CAPES. 5 July 2012Copyright: Ganesha Associates 2012 17
28 August 2012Ganesha Associates18 CAPES – scientific subdivisions –Ciências Biológicas I: Biologia Geral, Botânica, Genética, Oceanografia Biológica e Zoologia –Ciências Biológicas II: Biofísica, Biologia Molecular, Bioquímica, Farmacologia, Fisiologia, Morfologia, Anatomia, Histologia e Neurociências –Ciências Biológicas III: Microbiologia, Imunologia e Parasitologia –Saúde Coletiva –Ecologia e Meio Ambiente
28 August 2012Ganesha Associates19 Critérios de Classificação do Qualis Ciências Biológicas III: Microbiologia, Imunologia e Parasitologia QUALIS A - Trabalhos publicados em revista com fator de impacto (FI) igual ou maior que 1,945 QUALIS B - Trabalhos publicados em manuscrito com FI entre 1 e 1,945 trabalhos publicados nas Memórias do Instituto Oswaldo Cruz e no Brasilian Journal of Medical and Biological Research QUALIS C - trabalhos publicados em revistas nacionais ou internacionais com FI menor que 1.
28 August 2012Ganesha Associates20 Define: indexação A indexação de uma revista é o processo que confere o indicador de qualidade necessário para que esta possa pertencer a um ou mais bancos de dados de renome internacional, em uma área específica do conhecimento, como por exemplo, MEDLINE, ISI, LILACS, etc., segundo critérios preestabelecidos por essas instituições. É um fator importante para que a produção científica nela apresentada se torne conhecida e reconhecida internacionalmente. O periódico necessita, para isto, estar de acordo com os padrões básicos de apresentação formal e excelência de conteúdo científico, determinados internacionalmente.
28 August 2012Ganesha Associates21 As fontes de ‘indexação’ em ordem de ‘qualidade’ Google Scirus LILACS ScieLo PubMed/Medline Scopus Thomson Web of Science E muitos outros mais especializados
28 August 2012Ganesha Associates23 Journal selection process - ISI To properly reflect the global context in which scientific research takes place, and to provide balanced coverage in each category, ISI seeks to cover the best regional journals as well. However, rather than compare a regional journal with all journals in a particular subject category, the ISI editor considers it in terms of those journals in the category from the same geographic area. High journal publishing standards, especially timeliness, and English language bibliographic elements remain essential.
28 August 2012Ganesha Associates24 Journal selection process - Medline Scope and coverage: The journals brought to the Committee for review will contain articles predominantly on core biomedical subjects. –Journals whose content is predominantly a subject peripheral or related to biomedicine are occasionally brought to the Committee when they have some biomedical content. – In these cases, the Committee's advice is sought not only on the quality of the content but also on the contribution it makes to the coverage of the subjects in question. Generally, such journals will not be indexed if their biomedical content is already adequately covered. Quality of content: Scientific merit of a journal's content is the primary consideration in selecting journals for indexing. The validity, importance, originality, and contribution to the coverage of the field of the overall contents of each title are the key factors considered in recommending a title for indexing, whatever the intended purpose and audience.
28 August 2012Ganesha Associates25 Journal selection process - Medline Quality of editorial work: The journal should demonstrate features that contribute to the objectivity, credibility, and quality of its contents. These features may include information about the methods of selecting articles, especially on the explicit process of external peer review; statements indicating adherence to ethical guidelines; evidence that authors have disclosed financial conflicts of interest; timely correction of errata; explicit responsible retractions as appropriate; and opportunity for comments and dissenting opinion. Neither the advertising content nor commercial sponsorship should raise questions about the objectivity of the published material. Sponsorship by national or international professional societies may be considered. Production quality: Quality of the layout, printing, graphics, and illustrations are all considered in assessing a journal. Though not a requirement for selection, journals of archival importance should be printed on acid-free paper. For detailed information concerning the use of acid-free paper please refer to the Acid-Free Paper for Biomedical Literature Fact Sheet.Acid-Free Paper for Biomedical Literature
28 August 2012Ganesha Associates26 Journal selection process - Medline Foreign language journals: Criteria for selection are the same as for those written in English. In order to extend the accessibility of the journal’s content to a wider potential readership, the majority of published articles in the review issues must contain an English-language abstract before the title will be considered for possible indexing.
28 August 2012Ganesha Associates27 Journal selection process - SciELO Originality. Journals should publish the results of original scientific research which contributes significantly to knowledge in the area covered by the journal Peer review. The journal editor should specify how its peer review process will be applied for the approval of contributions Editorial board. The constitution of a journal's editorial board must be public. Its members should be renowned specialists from the country and abroad. Frequency. In the life sciences, a journal should be published at least four times a year and contain a minimum of 32 articles Lifetime. A journal must have published at least four issues. Timeliness of publication. A journal must be publishing on time according to its stated frequency. English. Journal articles must provide English translations of titles, abstracts, and keywords. Standardization. The journal must specify which standards it follows for editorial presentation of articles, e.g. bibliographical references style
28 August 2012Ganesha Associates30 The importance of ISI Impact Factors –What is a citation ? A citation for an article in a journal includes the author, title of article, title of periodical (also referred to as the source), volume and issue number, page number(s), and date of publication. A single citation is called a reference; a group of citations is a bibliography. –How are Impact Factors defined ? It is calculated each year by the commercial publisher Thomson for those journals which it tracks, and are published in the Journal Citation Report.Impact Impact Factors have a huge, but controversial, influence on the way published scientific research is perceived and evaluated. Originally intended to provide librarians with a relative index of importance for journals in a particular field in order to assist collection management
28 August 2012Ganesha Associates31 Impact Factor defined “The Impact Factor of a journal is probably the only quantitative way of assessing its worth and relevance to the academic community it serves.” Editorial, Journal of Computer-assisted Learning, 2001 Impact Factor 2011 = # citations received in 2011 to articles published in 2010/2009 ______________________________________________________ # number of source items published 2010 and 2009
28 August 2012Ganesha Associates32 Alternatives ? The Hirsch Index is rapidly gaining favor as a performance measurement tool for scientific authors. A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np - h) papers have no more than h citations Thus, the h-index reflects both the number of publications and the number of citations per publication. The index is designed to improve upon simpler measures such as the total number of citations or publications. Like the IF, the index works properly only for comparing scientists working in the same field. For more information see Scopus or Wikipedia
28 August 2012Ganesha Associates33 H-index - weaknesses The h-index is bounded by the total number of publications. This means that scientists with a short career are at an inherent disadvantage, regardless of the importance of their discoveries. For example, had Albert Einstein died in early 1906, his h-index would be stuck at 4 or 5, despite his being widely acknowledged as one of the most important physicists, even considering only his publications to that date.
28 August 2012Ganesha Associates34 Impact Factors and Authors Correlation between article citation rate and journal impact for four authors
28 August 2012Ganesha Associates35 Impact Factors and Journals Citation rates in 1986 or 1987 of articles published in three biochemical journals in 1983 or 1984, respectively. From Seglen, BMJ 1997;314:497
28 August 2012Ganesha Associates36 Impact Factors over time
28 August 2012Ganesha Associates37 Impact Factor and area
28 August 2012Ganesha Associates38 Impact Factors for Microbiology
28 August 2012Ganesha Associates39 Impact Factors for Neuroscience
28 August 2012Ganesha Associates40 Impact Factors and article type
28 August 2012Ganesha Associates41 Impact Factor - Summary points Use of journal impact factors conceals the difference in article citation rates Journals' impact factors are determined by technicalities unrelated to the scientific quality of their articles Journal impact factors depend on the research field: high impact factors are likely in journals covering large areas of basic research with a rapidly expanding but short lived literature that use many references per article Article citation rates determine the journal impact factor, not vice versa
28 August 2012Ganesha Associates42 The emergence of Open Access Clockwise: Harold Varmus, Michael Eisen, Pat Brown and David Lipman
28 August 2012Ganesha Associates43 Scientific publishing is a very profitable commercial business
28 August 2012Ganesha Associates44 Even when it is not-for-profit
28 August 2012Ganesha Associates45 Define: business model A business model (also called a business design) is the mechanism by which a business intends to generate revenue and profits. It is a summary of how a company plans to serve its customers. It involves both strategy and implementation Wikipedia
28 August 2012Ganesha Associates46 Exponential growth in the amount of data being published Subscription charge increases: 200% in last ten years - pressure on library budgets, reduction in number of subscriptions Publisher retention of copyright and therefore control “ Excessive” profits: 35%+ margins Restrictive online access - the bundle deal Some fundamental problems with the ‘print’ business model
28 August 2012Ganesha Associates47 Key features of the ‘open access’ business model Free, immediate access online Unrestricted use Author retains copyright Papers are deposited in a public online archive Author pays for publication
28 August 2012Ganesha Associates48 Advantages of Open Access Maximum impact for authors access to the largest possible audience New ways to access and use literature full-text searching and mining Greatly expanded access to research for scientists, educators, physicians, the public
28 August 2012Ganesha Associates49 Old ‘print’ subscription business model Researcher Publisher Reader Pay-per-viewSubscription Library Gov Funders Institutions Industry € €€€ €
28 August 2012Ganesha Associates50 New open access business model Researcher Publisher Reader € Public Digital Library Gov Funders Institutions Industry €
28 August 2012Ganesha Associates51 Open access a major force in Brasil
28 August 2012Ganesha Associates52 SciELO’s role in supporting OA
28 August 2012Ganesha Associates53 Consequences of Open Access A fundamental change in the business model underlying the scientific publishing industry Revenue: from subscription to fee-for-service Payer: was ‘the librarian’, now the research funder, usually the government, ultimately the tax-payer Marketing focus: was the librarian, now the researcher (author/reader) Future business driver: accountability to the tax payer
28 August 2012Ganesha Associates54 Some myths about Open Access Urban myth 1: “ISI doesn’t cover Open Access journals” ISI (Thomson Scientific) indexes around 200 Open Access journals (OAJs) for its Web of Science service. It applies the same criteria for inclusion in that index to Open Access journals as it does to subscription-based titles. Urban myth 2: “Open Access journals are not good quality” By dint of being included in the Web of Science index, around 200 Open Access journals enjoy the prestige of having an official Impact Factor, thus denoting some measure of quality to these titles. Moreover, some OAJs enjoy particularly high impact factors: the most notable in this respect, PLoS Biology, romped straight to the top position in the General Biology category the first year it qualified for such a calculation (journals must have been published for at least two years before an impact factor can be calculated). Urban myth 3: “Paying a publication fee to Open Access journals corrupts peer review” This would be very significant and deeply damaging to the integrity of the scholarly communication system were it true. It is not. Open Access journals that charge an article-processing fee have a firewall between the peer review process and the charge-imposition system, ensuring that authors cannot pay to subvert the peer review procedure in a manner akin to vanity publishing. Urban myth 4: “Open Access journal business models are unsustainable” Well, they may yet prove to be, but time is passing and we are seeing a move towards Open Access publishing not away from it, suggesting that some publishers at least are finding ways to sustain their publications even while making their content free online.. Alma Swan, American Scientist, 2006
28 August 2012Ganesha Associates55 What is Open Access not ? A movement – then who is the leader ? Free – electronic publishing, whatever the business model, is expensive Robust – benefits are being mis-sold to the main payer, and there is a real risk of loss of momentum A big issue for the majority of researchers - “…researchers do not consider that problems in accessing the journal literature are a significant barrier to their work…” Publishing Research Consortium 2006 Good news for national publishers in countries such as Brasil Just about documents and open archiving
28 August 2012Ganesha Associates56 Benefits of Open Access Access to full text. Better indexation and retrieval. Better integration between full text OA resources and other public archives of scientific data The potential for the emergence of new services based upon open access content or overlaying open access content (e.g. peer review services, text- and data-mining tools, bibliometrics and webometrics for better research evaluation, etc The potential for much wider access than the subscription publishing system gives for doctors/nurses, teachers/students, small biotech companies, etc Redistribution of publishing costs to richer nations Adapted from The Economic Impact of Enhanced Access to Research Findings by John Houghton & Peter Sheehan, Centre for Strategic Economic Studies, Victoria University, Melbourne, 2006
28 August 2012Ganesha Associates57 Big publishers fight back New Scientist recently published a plan by PR firm Dezenhall Resources detailing how traditional science publishers could turn government opinion against open-access journals.a plan There’s nothing wrong with employing a famously tough PR firm, but the memo reveals that Dezenhall recommended sidestepping the real issues around peer review and the business model that powers it, in favor of “rhetorical campaign points” about Big Government and scientists’ hypocrisy.famously toughpeer reviewthe business model The memo notes that the key issue is that “publishers are trying to protect their businesses and the integrity of the research they publish.” Yet, two of Dezenhall’s example messages focused on government “censorship” and “nationalization” of science. No one really believes, on either side, that government is taking over science publishing. The catalytic bill, Federal Research Public Access Act (S.2695), would only require that articles based on federally funded research be opened to public inspection within 6 months of publication.Federal Research Public Access Act
28 August 2012Ganesha Associates58 Impact factors drive submissions, (PLOS) First impact factor announced
28 August 2012Ganesha Associates59 Web access drives impact factor
28 August 2012Ganesha Associates60 Open Access – beyond the document Synthesize evidence systematically from many studies and teams of investigators and anticipate this integration from the design phase of research. –John Ioannides, NIH 2006 “Upload and share your raw data, and have a high impact factor for your blog — or perish? That day has not yet come, but web technologies, from personal publishing tools such as blogs to electronic laboratory notebooks, are pushing the character of the web from that of a large library towards providing a user-driven collaborative workspace”, –Nature editorial, Dec 2005 “…the 48% of trials which made their data available received 85% of the total citations… –Heather Piowar, March 2007
28 August 2012Ganesha Associates61 Open Archiving ACS has re-announced its Author Choice model to draw attention to the fact that authors can now automatically self-archive through payment of the Author Choice fee. Elsevier has announced a deal with the Howard Hughes Medical Institute (HHMI) which allows HHMI to pay a fee to have all of its funded research results which happen to be published in Elsevier journals automatically archived in PubMed six months after publication. In both instances the publisher has taken the lead, as arguably publishers historically always have, to ensure accessibility. In both instances, the idea of fee for service is maintained in the relation - the deals reduce bureaucratic administration costs and increases efficiency: Elsevier got $1,000 for every Cell Press article, and $1,500 for each article from other Elsevier journals. In general terms, the service encourages researchers with HHMI grants to dispatch their "obligations? to Open Access in this painless way, distracts them from other thoughts about institutional repositories, and encourages the use of Elsevier journals for publication. Presumably therefore other publishers will follow suit with HHMI. David Worlock, EPS, April 2007
28 August 2012Ganesha Associates62 Performance of OA medical journals (ISI)