RSS 2.0
Critical Assessment of Information Extraction in Biology - data sets are available from Resources/Corpora and require registration.

BioCreative V

Call for Participation (Events) [2015-03-31]

BioCreative V Challenge and Workshop

September 9-11, 2015
Sevilla, Spain

BioCreative: Critical Assessment of Information Extraction in Biology is a community-wide effort for evaluating text mining and information extraction systems applied to the biological domain. Built on the success of the previous BioCreative Challenge Evaluations and Workshops (BioCreative I, II, II.5, III, 2012 workshop, and IV) [1-6] the BioCreative Organizing Committee will host the BioCreative V Challenge in Sevilla, Spain on September 9-11, 2015. One key goal of BioCreative is the active involvement of the text mining user community in the design of the tracks, preparation of corpus and the testing of interactive systems. For BioCreative V, the selection of the tracks has been driven in part by suggestions from the biocuration community, by our goal of addressing interoperability -- a major barrier to adoption to text mining tools --, and via an open call for tasks of interest to both the bioNLP and user communities.

BioCreative V will consist of five tracks. Teams can participate in one or more of these tracks. Team registration will start on January 1, 2015 and will continue until final commitment is requested by the individual tracks.

To register a team go to

Funds are available for US participants to attend BioCreative 5. Information on how to apply will follow. Women, under-represented minorities, students, and post-doctoral fellows are encouraged to apply.

  • Track 1: Collaborative Biocurator Assistant Task (BioC) – Development of BioC-compatible modules which complement each other and an integrated system that assists BioGRID curators. A non-competitive, cooperative task in which participants work together to build a better system.
    Organizers:Sun Kim, Donald Comeau, Rezarta Islamaj Dogan, Andrew Chatr-aryamontri and W. John Wilbur
    Important Dates
    March 2015: Deadline for participation
    Mar-May. 2015: Developing individual systems and iterative system integration
    June 2015: Individual systems due date
    June-July 2015: Overall system evaluation
    August 2015: Manuscript deadline for meeting proceedings

  • Track 2: CHEMDNER patents - Identification of chemical compounds and of relevant biological context in patents.
    Organizers:Martin Krallinger, Alfonso Valencia, Florian Leitner, Obdulia Rabal and Julien Oyarzabal
    Important Dates
    February 2015: Sample set release together with annotation guidelines
    April 2015: Training set release
    May 2015: Development set release
    July 2015: Test set release
    August 2015: The evaluated results returned to the participants
    beginning of September 2015: The camera ready system description write ups

  • Track 3: Chemical-disease relation (CDR) task - Automatic detection of chemical/drugs and diseases, and their relations in PubMed abstracts. In particular, the CDR task focuses on extracting the relationship of drug-induced diseases.
    Organizers:Zhiyong Lu and Thomas Wiegers
    Important Dates
    February-March 2015: Release of training data
    April-May 2015: Release of development data
    July 2015: Release of test data
    July-August 2015: Team submissions due
    August-Early September, 2015: Team workshop paper due

  • Track 4: Extraction of causal network information in Biological Expression Language (BEL) - Text mining solutions to develop and test novel approaches for relation extraction in the context of pathway networks. The goal is to assess the utility of such tools for the automated annotation and network expansion, and their suitability as supporting tools for assisted curation.
    Organizers:Fabio Rinaldi, Sam Ansari and Juliane Fluck
    Important Dates
    February 15, 2015: Release sample data
    February 28, 2015: Release training data
    June 14, 2015: Release test data
    June 16, 2015: Submission of results deadline
    July 10, 2015: Delivery of evaluation results
    July 20, 2015: Paper submission
    August 15, 2015: Camera-ready
    Please note that all dates are indicative only and subject to change

  • Track 5: Interactive Curation (IAT): Demonstration and evaluation of web-based systems addressing user-defined tasks, evaluated by curators on performance and usability.
    Organizers:Catalina Oana Tudor, Qinghua Wang and Cecilia Arighi
    Important Dates
    January 31, 2015: Submission of letter of intent by teams
    March 31, 2015: Submission of system document by teams
    April 10, 2015: Notification of accepted systems
    April-May 2015: Systems development
    Mid-June 2015: Individual systems ready for user testing
    June-July 2015: Overall system evaluation by curators
    August 2015: Manuscript deadline for meeting proceedings
  • Back to top


  • Cecilia Arighi, University of Delaware, USA
  • Kevin Cohen, University of Colorado, USA
  • Donal Comeau, National Center for Biotechnology Information (NCBI), NIH, USA
  • Juliane Fluck, Fraunhofer Institute for Algorithms and Scientific Computing SCAI, Germany
  • Rezarta Islamaj Dogan, National Center for Biotechnology Information (NCBI), NIH, USA
  • Lynette Hirschman, MITRE Corporation, USA
  • Sun Kim, National Center for Biotechnology Information (NCBI), NIH, USA
  • Martin Krallinger, Spanish National Cancer Centre, CNIO, Spain
  • Zhiyong Lu, National Center for Biotechnology Information (NCBI), NIH, USA
  • Fabio Rinaldi, Institute of Computational Linguistics, University of Zurich, Switzerland
  • Alfonso Valencia, Spanish National Cancer Centre, CNIO, Spain
  • Thomas Wiegers, North Carolina State University, USA
  • John Wilbur, National Center for Biotechnology Information (NCBI), NIH, USA
  • Cathy Wu, University of Delaware and Georgetown University, USA
  • Back to top

    BioCreative is partially funded by NIH/NIGMS R13-GM109648-01A1


    1. Hirschman, L., A. Yeh, C. Blaschke, and A. Valencia, Overview of BioCreAtIvE: critical assessment of information extraction for biology. BMC Bioinformatics, 2005. 6 Suppl 1: p. S1. PMCID:PMC1869002
    2. Krallinger, M., A. Morgan, L. Smith, F. Leitner, L. Tanabe, J. Wilbur, L. Hirschman, and A. Valencia, Evaluation of text-mining systems for biology: overview of the Second BioCreative community challenge. Genome Biol, 2008. 9 Suppl 2: p. S1. PMCID:PMC2559980
    3. Leitner, F., S.A. Mardis, M. Krallinger, G. Cesareni, L.A. Hirschman, and A. Valencia, An Overview of BioCreative II.5. IEEE/ACM Trans Comput Biol Bioinform, 2010. 7(3): p. 385-99.
    4. Arighi, C.N., Z. Lu, M. Krallinger, K.B. Cohen, J. Wilbur, A. Valencia, L. Hirschman, and C.H. Wu, Overview of the BioCreative III Workshop BMC Bioinformatics, 2011. 12 Suppl. 8: p. S1.
    5. Wu CH, Arighi CN, Cohen KB, Hirschman L, Krallinger Martin, Lu Z, Mattingly C, Valencia A, Wiegers TC, Wilbur WJ: Editorial: BioCreative-2012 Virtual Issue. Database (Oxford) 2012:bas049.PMC3514749.
    6. Arighi CN, Wu CH, Cohen KB, Hirschman L, Krallinger M, Valencia A, Lu Z,Wilbur JW, Wiegers TC. BioCreative-IV virtual issue. Database (Oxford). 2014 May 22;2014. PMCID: PMC4030502.

    Back to top


    BC IV - CHEMDNER: Journal of Cheminformatics special issue (Resources) [2015-03-31]

    BC IV - CHEMDNER: Journal of Cheminformatics special issue

    The entire special issue is available at: here.

    Table of Contents

    1. CHEMDNER: The drugs and chemical names extraction challenge Martin Krallinger, Florian Leitner, Obdulia Rabal, Miguel Vazquez, Julen Oyarzabal, Alfonso Valencia. Journal of Cheminformatics 2015, 7(Suppl 1):S1 (19 January 2015)[ PUBMED]
    2. The CHEMDNER corpus of chemicals and drugs and its annotation principles Martin Krallinger, Obdulia Rabal, Florian Leitner, Miguel Vazquez, David Salgado, Zhiyong Lu, Robert Leaman, Yanan Lu, Donghong Ji, Daniel M Lowe, Roger A Sayle, Riza Batista-Navarro, Rafal Rak, Torsten Huber, Tim Rocktäschel, Sérgio Matos, David Campos, Buzhou Tang, Hua Xu, Tsendsuren Munkhdalai, Keun Ryu, SV Ramanan, Senthil Nathan, Slavko Žitnik, Marko Bajec, Lutz Weber, Matthias Irmer, Saber A Akhondi, Jan A Kors, Shuo Xu, Xin An, Utpal Sikdar, Asif Ekbal, Masaharu Yoshioka, Thaer M Dieb, Miji Choi, Karin Verspoor, Madian Khabsa, C Giles, Hongfang Liu, Komandur Ravikumar, Andre Lamurias, Francisco M Couto, Hong-Jie Dai, Richard Tsai, Caglar Ata, Tolga Can, Anabel Usié, Rui Alves, Isabel Segura-Bedmar, Paloma Martínez, Julen Oyarzabal, Alfonso Valencia et al. Journal of Cheminformatics 2015, 7(Suppl 1):S2 (19 January 2015) [ PUBMED]
    3. tmChem: a high performance approach for chemical named entity recognition and normalization Robert Leaman, Chih-Hsuan Wei, Zhiyong Lu Journal of Cheminformatics 2015, 7(Suppl 1):S3 (19 January 2015) [ PUBMED]
    4. CHEMDNER system with mixed conditional random fields and multi-scale word clustering Yanan Lu, Donghong Ji, Xiaoyuan Yao, Xiaomei Wei, Xiaohui Liang Journal of Cheminformatics 2015, 7(Suppl 1):S4 (19 January 2015) [ PUBMED]
    5. LeadMine: a grammar and dictionary driven approach to entity recognition Daniel M Lowe, Roger A Sayle Journal of Cheminformatics 2015, 7(Suppl 1):S5 (19 January 2015) [ PUBMED]
    6. Optimising chemical named entity recognition with pre-processing analytics, knowledge-rich features and heuristics Riza Batista-Navarro, Rafal Rak, Sophia Ananiadou Journal of Cheminformatics 2015, 7(Suppl 1):S6 (19 January 2015) [ PUBMED]
    7. A document processing pipeline for annotating chemical entities in scientific documents David Campos, Sérgio Matos, José L Oliveira Journal of Cheminformatics 2015, 7(Suppl 1):S7 (19 January 2015) [ PUBMED]
    8. A comparison of conditional random fields and structured support vector machines for chemical entity recognition in biomedical literature Buzhou Tang, Yudong Feng, Xiaolong Wang, Yonghui Wu, Yaoyun Zhang, Min Jiang, Jingqi Wang, Hua Xu Journal of Cheminformatics 2015, 7(Suppl 1):S8 (19 January 2015) [ PUBMED]
    9. Incorporating domain knowledge in chemical and biomedical named entity recognition with word representations Tsendsuren Munkhdalai, Meijing Li, Khuyagbaatar Batsuren, Hyeon Park, Nak Choi, Keun Ryu Journal of Cheminformatics 2015, 7(Suppl 1):S9 (19 January 2015) [ PUBMED]
    10. Recognition of chemical entities: combining dictionary-based and grammar-based approaches Saber A Akhondi, Kristina M Hettne, Eelke van der Horst, Erik M van Mulligen, Jan A Kors Journal of Cheminformatics 2015, 7(Suppl 1):S10 (19 January 2015) [ PUBMED]
    11. A CRF-based system for recognizing chemical entity mentions (CEMs) in biomedical literature Shuo Xu, Xin An, Lijun Zhu, Yunliang Zhang, Haodong Zhang Journal of Cheminformatics 2015, 7(Suppl 1):S11 (19 January 2015) [ PUBMED]
    12. Chemical entity extraction using CRF and an ensemble of extractors Madian Khabsa, C Giles Journal of Cheminformatics 2015, 7(Suppl 1):S12 (19 January 2015) [ PUBMED]
    13. Improving chemical entity recognition through h-index based semantic similarity Andre Lamurias, João D Ferreira, Francisco M Couto Journal of Cheminformatics 2015, 7(Suppl 1):S13 (19 January 2015) [ PUBMED]
    14. Enhancing of chemical compound and drug name recognition using representative tag scheme and fine-grained tokenization Hong-Jie Dai, Po-Ting Lai, Yung-Chun Chang, Richard Tsai Journal of Cheminformatics 2015, 7(Suppl 1):S14 (19 January 2015) [ PUBMED]
    15. CheNER: a tool for the identification of chemical entities and their classes in biomedical literature Anabel Usié, Joaquim Cruz, Jorge Comas, Francesc Solsona, Rui Alves Journal of Cheminformatics 2015, 7(Suppl 1):S15 (19 January 2015) [ PUBMED]

    BioCreative V

    BioCreative Call for Task Proposals and Panels (Events) [2014-07-30]

    BioCreative invites task and panel proposal submissions in preparation for the 2015 workshop, to be held September 9-11th, 2015 in Sevilla, Spain.

    BioCreative challenges consist of a community-wide effort to evaluate text mining and information extraction systems applied to the biological domain. The main emphasis is on the comparison of methods and the community assessment of scientific progress, rather than on the purely competitive aspects. In the past ten years BioCreative challenges have spanned a number of tasks particularly important to biocuration for the effective extraction of biologically relevant information from the literature

    In order to better serve the community, we are soliciting proposals from the larger community for new BioCreative Tasks and Panel Sessions.

    A BioCreative panel proposal would highlight an interesting research topic that might result in useful applications for end-users from the biomedical and bioinformatics domain, and could potentially translate to a task in future BioCreative challenges. Click here for more information.

    A BioCreative task proposal would outline a ready to run text mining task whose results should be reported in the BioCreative V workshop. Click here for more information.

    Submission Deadline is October 15, 2014.

    1-BioCreative Call for Workshop Panel Proposals

    The BioCreative (Critical Assessment of Information Extraction in Biology) evaluation Workshops offer the opportunity to discuss the results of cutting-edge text mining and information extraction strategies applied to a set of tasks relevant to the biomedical domain. A unique characteristic of this effort is its collaborative and interdisciplinary nature, bringing together experts from various fields, including Text Mining and Natural Language Processing, biocuration, publishing houses and bioinformatics.

    In order to highlight interesting research topics that might result in useful applications for end-users from the biomedical and bioinformatics domain, the BioCreative workshops include panel sessions with invited experts that can contribute to better characterize the user needs and design potential new tasks.

    Past panels have included publisher perspectives, approaches to system interoperability and applications of text mining to metagenomics.

    In this opportunity, we are inviting the community to submit a 1-page proposal for a workshop panel for BioCreative V, to be held in September 2015, Sevilla, Spain.

    Proposals will be reviewed by the BioCreative Steering Committee and decisions will be given in a timely manner. Criteria for acceptance of proposals include the following:

    • The proposal addresses a research area of interest to a significant number of end users with a clear need for integrating text mining systems to improve the analysis of and access to relevant information.
    • The information that needs to be integrated for the topic of interest must be contained in publicly accessible data repositories.
    • A list of international experts willing to commit to attending the workshop and presenting their perspective on the area of interest should be provided.
    • A tentative list of discussion topics should be included which could be covered in a discussion panel of one hour.

    Proposal Submission

    We request that you submit your proposals by October 15, 2014, by electronic mail (no postal submissions) to: with Subject BioCreative V Panel proposal.

    Back to top

    2-BioCreative Call for Task Proposals

    BioCreative invites task proposals for the 2015 workshop, to be held September, 2015 in Sevilla, Spain. Specifically, we are soliciting proposals from the larger community for new BioCreative Tasks.

    We request that you submit your proposals by October 15, 2014, by electronic mail (no postal submissions) to: with the subject line “BioCreative V task proposal”. You can also direct any inquiries concerning this process to the same email address.

    Task Proposal Content

    Task proposals for BioCreative 2015 should be between two and four pages in length, and should contain the following information:

    • Title of the task
    • Task description: A short description of the task and a justification of why the proposed task is appropriate for BioCreative. Discuss why the task is timely and important, and how the task is being addressed now in the field. If possible, provide a baseline performance using an existing method to help assess the difficulty of the proposed task. Address both background and novelty. The first issue concerns related work, other evaluations, and published systems that do something similar, while the second issue relates to important aspects that have not been addressed so far.
    • Task data: A short description of the data detailing the source and size of training, development, and test data. The data should be publicly available, and use of BioC as an interchange format is preferred. Ideally, there should exist curated data resources that can be used as the basis for developing training data, along with annotation guidelines. If available, provide a sample annotated corpus; if data is not yet fully annotated, include explicit plans for its completion (e.g. annotators, annotation tool, etc.).
    • Task Evaluation: A short description specifying how the submitted results will be evaluated, the proposed metrics for evaluation and whether the software to perform the evaluation exists; if not, specify who will develop the evaluation software.
    • Task Timeline: Provide a suggested timeline for completion of the task, including preparation of training data, release of training and test data, evaluation and system write ups.
    • Task organizing committee: Describe the team, and their prior experience.
    • Task beneficiaries: State what community this task is relevant to, and how they will benefit from the tools/capabilities developed through the task. Explain how this community would participate or contribute to the task. For an interactive task, suggest communities who would be candidate participant users.
    • Task participants: Suggest potential participants for the task.

    Proposal Evaluation Criteria

    The Biocreative Workshops generally consist of five specific tasks. Proposals will be reviewed by the Steering Committee and decisions will be given in a timely manner. Criteria for acceptance of proposals include the following:

    • Perceived interest to the BioCreative community
    • Design of the task
    • Appropriate number of participants
    • An appropriate organizing committee
    • Lack of such activities on the topic on other venues, or the appropriateness of bringing this activity to the BioCreative venue.

    Accepted proposals will be advertised on the BioCreative website, and other regular mailing lists. Proposals should be submitted as soon as possible but no later than October 15, 2014. Notifications of acceptance or rejection will be sent to submitters around November 15, 2014. Task organizers will be included in the BioCreative Organizing Committee.

    The task organizers will be responsible for:

    • Writing task descriptions for the BioCreative workshop website, calls for participation and registration, which BioCreative will distribute to its membership
    • Advertising the task, especially to potential audiences outside the BioCreative community
    • Reviewing requests to participate in the task and determining task participants
    • Scheduling the activities of the task
    • Submitting accepted author lists and papers for the BioCreative workshop proceedings.
    • Participating in BioCreative Organizing Committee meetings (typically every 2-3 weeks)

    The BioCreative Organizing Committee will provide logistical support, will take care of all local arrangements, and will arrange for reproducing and distributing the proceedings, as well as arranging for a special issue.

    Back to top