RSS 2.0
Critical Assessment of Information Extraction in Biology - data sets are available from Resources/Corpora and require registration.

BioCreative V

Call for Participation (Events) [2014-12-19]

BioCreative V Challenge and Workshop

September 9-11, 2015
Sevilla, Spain

BioCreative: Critical Assessment of Information Extraction in Biology is a community-wide effort for evaluating text mining and information extraction systems applied to the biological domain. Built on the success of the previous BioCreative Challenge Evaluations and Workshops (BioCreative I, II, II.5, III, 2012 workshop, and IV) [1-6] the BioCreative Organizing Committee will host the BioCreative V Challenge in Sevilla, Spain on September 9-11, 2015. One key goal of BioCreative is the active involvement of the text mining user community in the design of the tracks, preparation of corpus and the testing of interactive systems. For BioCreative V, the selection of the tracks has been driven in part by suggestions from the biocuration community, by our goal of addressing interoperability -- a major barrier to adoption to text mining tools --, and via an open call for tasks of interest to both the bioNLP and user communities.

BioCreative V will consist of five tracks. Teams can participate in one or more of these tracks. Team registration will start on January 1, 2015 and will continue until final commitment is requested by the individual tracks.

To register a team go to http://www.biocreative.org/events/biocreative-v/biocreative-v-team/.

Funds are available for US participants to attend BioCreative 5. Information on how to apply will follow. Women, under-represented minorities, students, and post-doctoral fellows are encouraged to apply.

  • Track 1: Collaborative Biocurator Assistant Task (BioC) – Development of BioC-compatible modules which complement each other and an integrated system that assists BioGRID curators. A non-competitive, cooperative task in which participants work together to build a better system.
    Organizers:Sun Kim, Donald Comeau, Rezarta Islamaj Dogan, Andrew Chatr-aryamontri and W. John Wilbur
    Important Dates
    March 2015: Deadline for participation
    Mar-May. 2015: Developing individual systems and iterative system integration
    June 2015: Individual systems due date
    June-July 2015: Overall system evaluation
    August 2015: Manuscript deadline for meeting proceedings
    URL: http://www.biocreative.org/tasks/biocreative-v/track-1-bioc/

  • Track 2: CHEMDNER patents - Identification of chemical compounds and of relevant biological context in patents.
    Organizers:Martin Krallinger, Alfonso Valencia, Florian Leitner, Obdulia Rabal and Julien Oyarzabal
    Important Dates
    February 2015: Sample set release together with annotation guidelines
    April 2015: Training set release
    May 2015: Development set release
    July 2015: Test set release
    August 2015: The evaluated results returned to the participants
    beginning of September 2015: The camera ready system description write ups
    URL: http://www.biocreative.org/tasks/biocreative-v/track-2-chemdner/

  • Track 3: Chemical-disease relation (CDR) task - Automatic detection of chemical/drugs and diseases, and their relations in PubMed abstracts. In particular, the CDR task focuses on extracting the relationship of drug-induced diseases.
    Organizers:Zhiyong Lu and Thomas Wiegers
    Important Dates
    February-March 2015: Release of training data
    April-May 2015: Release of development data
    July 2015: Release of test data
    July-August 2015: Team submissions due
    August-Early September, 2015: Team workshop paper due
    URL: http://www.biocreative.org/tasks/biocreative-v/track-3-cdr/

  • Track 4: Extraction of causal network information in Biological Expression Language (BEL) - Text mining solutions to develop and test novel approaches for relation extraction in the context of pathway networks. The goal is to assess the utility of such tools for the automated annotation and network expansion, and their suitability as supporting tools for assisted curation.
    Organizers:Fabio Rinaldi, Sam Ansari and Juliane Fluck
    Important Dates
    February 15, 2015: Release training data
    April 27, 2015: Release test data
    April 30, 2015: Submission of results deadline
    June 15, 2015: Delivery of evaluation results
    July 1, 2015: Paper submission
    August 15, 2015: Camera-ready
    URL: http://www.biocreative.org/tasks/biocreative-v/track-4-bel-task/

  • Track 5: Interactive Curation (IAT): Demonstration and evaluation of web-based systems addressing user-defined tasks, evaluated by curators on performance and usability.
    Organizers:Catalina Oana Tudor and Cecilia Arighi
    Important Dates
    January 31, 2015: Submission of letter of intent by teams
    March 1, 2015: Submission of system document by teams
    March 15, 2015: Notification of accepted systems
    March-May 2015: Systems development
    Mid-June 2015: Individual systems ready for user testing
    June-July 2015: Overall system evaluation by curators
    August 2015: Manuscript deadline for meeting proceedings
    URL: http://www.biocreative.org/tasks/biocreative-v/track-user-interactive-task/
  • Back to top


    BIOCREATIVE ORGANIZING COMMITTEE

  • Cecilia Arighi, University of Delaware, USA
  • Kevin Cohen, University of Colorado, USA
  • Donal Comeau, National Center for Biotechnology Information (NCBI), NIH, USA
  • Rezarta Islamaj Dogan, National Center for Biotechnology Information (NCBI), NIH, USA
  • Lynette Hirschman, MITRE Corporation, USA
  • Sun Kim, National Center for Biotechnology Information (NCBI), NIH, USA
  • Martin Krallinger, Spanish National Cancer Centre, CNIO, Spain
  • Zhiyong Lu, National Center for Biotechnology Information (NCBI), NIH, USA
  • Fabio Rinaldi, Institute of Computational Linguistics, University of Zurich, Switzerland
  • Alfonso Valencia, Spanish National Cancer Centre, CNIO, Spain
  • Thomas Wiegers, North Carolina State University, USA
  • John Wilbur, National Center for Biotechnology Information (NCBI), NIH, USA
  • Cathy Wu, University of Delaware and Georgetown University, USA
  • Back to top

    REFERENCES

    1. Hirschman, L., A. Yeh, C. Blaschke, and A. Valencia, Overview of BioCreAtIvE: critical assessment of information extraction for biology. BMC Bioinformatics, 2005. 6 Suppl 1: p. S1. PMCID:PMC1869002
    2. Krallinger, M., A. Morgan, L. Smith, F. Leitner, L. Tanabe, J. Wilbur, L. Hirschman, and A. Valencia, Evaluation of text-mining systems for biology: overview of the Second BioCreative community challenge. Genome Biol, 2008. 9 Suppl 2: p. S1. PMCID:PMC2559980
    3. Leitner, F., S.A. Mardis, M. Krallinger, G. Cesareni, L.A. Hirschman, and A. Valencia, An Overview of BioCreative II.5. IEEE/ACM Trans Comput Biol Bioinform, 2010. 7(3): p. 385-99.
    4. Arighi, C.N., Z. Lu, M. Krallinger, K.B. Cohen, J. Wilbur, A. Valencia, L. Hirschman, and C.H. Wu, Overview of the BioCreative III Workshop BMC Bioinformatics, 2011. 12 Suppl. 8: p. S1.
    5. Wu CH, Arighi CN, Cohen KB, Hirschman L, Krallinger Martin, Lu Z, Mattingly C, Valencia A, Wiegers TC, Wilbur WJ: Editorial: BioCreative-2012 Virtual Issue. Database (Oxford) 2012:bas049.PMC3514749.
    6. Arighi CN, Wu CH, Cohen KB, Hirschman L, Krallinger M, Valencia A, Lu Z,Wilbur JW, Wiegers TC. BioCreative-IV virtual issue. Database (Oxford). 2014 May 22;2014. PMCID: PMC4030502.

    Back to top

    BioCreative Call for Task Proposals and Panels (Events) [2014-07-30]

    BioCreative invites task and panel proposal submissions in preparation for the 2015 workshop, to be held September 9-11th, 2015 in Sevilla, Spain.

    BioCreative challenges consist of a community-wide effort to evaluate text mining and information extraction systems applied to the biological domain. The main emphasis is on the comparison of methods and the community assessment of scientific progress, rather than on the purely competitive aspects. In the past ten years BioCreative challenges have spanned a number of tasks particularly important to biocuration for the effective extraction of biologically relevant information from the literature

    In order to better serve the community, we are soliciting proposals from the larger community for new BioCreative Tasks and Panel Sessions.

    A BioCreative panel proposal would highlight an interesting research topic that might result in useful applications for end-users from the biomedical and bioinformatics domain, and could potentially translate to a task in future BioCreative challenges. Click here for more information.

    A BioCreative task proposal would outline a ready to run text mining task whose results should be reported in the BioCreative V workshop. Click here for more information.

    Submission Deadline is October 15, 2014.

    1-BioCreative Call for Workshop Panel Proposals

    The BioCreative (Critical Assessment of Information Extraction in Biology) evaluation Workshops offer the opportunity to discuss the results of cutting-edge text mining and information extraction strategies applied to a set of tasks relevant to the biomedical domain. A unique characteristic of this effort is its collaborative and interdisciplinary nature, bringing together experts from various fields, including Text Mining and Natural Language Processing, biocuration, publishing houses and bioinformatics.

    In order to highlight interesting research topics that might result in useful applications for end-users from the biomedical and bioinformatics domain, the BioCreative workshops include panel sessions with invited experts that can contribute to better characterize the user needs and design potential new tasks.

    Past panels have included publisher perspectives, approaches to system interoperability and applications of text mining to metagenomics.

    In this opportunity, we are inviting the community to submit a 1-page proposal for a workshop panel for BioCreative V, to be held in September 2015, Sevilla, Spain.

    Proposals will be reviewed by the BioCreative Steering Committee and decisions will be given in a timely manner. Criteria for acceptance of proposals include the following:

    • The proposal addresses a research area of interest to a significant number of end users with a clear need for integrating text mining systems to improve the analysis of and access to relevant information.
    • The information that needs to be integrated for the topic of interest must be contained in publicly accessible data repositories.
    • A list of international experts willing to commit to attending the workshop and presenting their perspective on the area of interest should be provided.
    • A tentative list of discussion topics should be included which could be covered in a discussion panel of one hour.

    Proposal Submission

    We request that you submit your proposals by October 15, 2014, by electronic mail (no postal submissions) to: krallinger.martin@gmail.com with Subject BioCreative V Panel proposal.

    Back to top

    2-BioCreative Call for Task Proposals

    BioCreative invites task proposals for the 2015 workshop, to be held September, 2015 in Sevilla, Spain. Specifically, we are soliciting proposals from the larger community for new BioCreative Tasks.

    We request that you submit your proposals by October 15, 2014, by electronic mail (no postal submissions) to: Rezarta.Islamaj@nih.gov with the subject line “BioCreative V task proposal”. You can also direct any inquiries concerning this process to the same email address.

    Task Proposal Content

    Task proposals for BioCreative 2015 should be between two and four pages in length, and should contain the following information:

    • Title of the task
    • Task description: A short description of the task and a justification of why the proposed task is appropriate for BioCreative. Discuss why the task is timely and important, and how the task is being addressed now in the field. If possible, provide a baseline performance using an existing method to help assess the difficulty of the proposed task. Address both background and novelty. The first issue concerns related work, other evaluations, and published systems that do something similar, while the second issue relates to important aspects that have not been addressed so far.
    • Task data: A short description of the data detailing the source and size of training, development, and test data. The data should be publicly available, and use of BioC as an interchange format is preferred. Ideally, there should exist curated data resources that can be used as the basis for developing training data, along with annotation guidelines. If available, provide a sample annotated corpus; if data is not yet fully annotated, include explicit plans for its completion (e.g. annotators, annotation tool, etc.).
    • Task Evaluation: A short description specifying how the submitted results will be evaluated, the proposed metrics for evaluation and whether the software to perform the evaluation exists; if not, specify who will develop the evaluation software.
    • Task Timeline: Provide a suggested timeline for completion of the task, including preparation of training data, release of training and test data, evaluation and system write ups.
    • Task organizing committee: Describe the team, and their prior experience.
    • Task beneficiaries: State what community this task is relevant to, and how they will benefit from the tools/capabilities developed through the task. Explain how this community would participate or contribute to the task. For an interactive task, suggest communities who would be candidate participant users.
    • Task participants: Suggest potential participants for the task.

    Proposal Evaluation Criteria

    The Biocreative Workshops generally consist of five specific tasks. Proposals will be reviewed by the Steering Committee and decisions will be given in a timely manner. Criteria for acceptance of proposals include the following:

    • Perceived interest to the BioCreative community
    • Design of the task
    • Appropriate number of participants
    • An appropriate organizing committee
    • Lack of such activities on the topic on other venues, or the appropriateness of bringing this activity to the BioCreative venue.

    Accepted proposals will be advertised on the BioCreative website, and other regular mailing lists. Proposals should be submitted as soon as possible but no later than October 15, 2014. Notifications of acceptance or rejection will be sent to submitters around November 15, 2014. Task organizers will be included in the BioCreative Organizing Committee.

    The task organizers will be responsible for:

    • Writing task descriptions for the BioCreative workshop website, calls for participation and registration, which BioCreative will distribute to its membership
    • Advertising the task, especially to potential audiences outside the BioCreative community
    • Reviewing requests to participate in the task and determining task participants
    • Scheduling the activities of the task
    • Submitting accepted author lists and papers for the BioCreative workshop proceedings.
    • Participating in BioCreative Organizing Committee meetings (typically every 2-3 weeks)

    The BioCreative Organizing Committee will provide logistical support, will take care of all local arrangements, and will arrange for reproducing and distributing the proceedings, as well as arranging for a special issue.

    Back to top

    BCBioCuration2014

    Workshop 1 - BioCreative Workshop on Text Mining Applications (Events) [2014-03-26]

    Biocuration 2014 Conference at the University of Toronto, Toronto, Canada
    Monday April 7, 3-5pm, 2014. East Common Room, Hart House.
    Chairs: Cecilia Arighi (1) and Lynette Hirschman (2)
    1 Center for Bioinformatics and Computational Biology, University of Delaware, DE, USA
    2 The MITRE Corporation, Bedford, MA, USA
    Agenda

    BioCreative: Critical Assessment of Information Extraction in Biology is an international community-wide effort that evaluates text mining (TM) and information extraction systems applied to the biomedical domain (http://www.biocreative.org/). A unique characteristic of this effort is its collaborative and interdisciplinary nature, bringing together experts from various fields, including TM, biocuration, publishing houses and bioinformatics. The aim of this workshop is to demonstrate advances in the application of TM systems, and encourage active involvement of users in guiding TM system development and adoption [1-5].

    The topics that will be presented include:
    1) The BioCreative Interoperability Initiative: the BioC format has been proposed as a simple extensible mark-up language format to share text documents and annotations. The annotation approach allows the representation of a large number of different annotations to support a variety of applications [6].
    2) TM and its users: in this section a brief overview of the BioCreative user interactive task will be presented [7-9] followed by short demos of selected TM systems, a user perspective on current needs and applications of TM tools, and engagement of new communities such as Metagenomics.

    Workshop agenda
    Presentations

  • Welcome to workshop, Lynette Hirschman (5 min)
  • Presentations and perspectives, panelists/presenters:
    • a. Interoperability:
      BioC: a minimalist approach to interoperability for biomedical text processing
      Donald Comeau, National Center for Biotechnology Information, National Library of Medicine (15 min)
      b. Text Mining and its users:
      The user interactive task in BioCreative Challenges
      Cecilia Arighi, Center for Bioinformatics and Computational Biology, University of Delaware (15 min)
      BioQRator: a web-based interactive biomedical literature curating system
      Don Comeau, National Center for Biotechnology Information, National Library of Medicine (10 min)
      eCuration: speed curating with PubTator
      Zhiyong Lu, National Center for Biotechnology Information, National Library of Medicine (10 min)
      Semi-automated extraction of experimental methods for assisted curation of RegulonDB
      Fabio Rinaldi, Institute of Computational Linguistics, University of Zurich (10 min)
      The use of text-mining tools during literature triage and functional annotation in the Mouse Genome Database
      Harold Drabkin, MGI, The Jackson Laboratory (10 min)
      Text mining and Publishers
      Bartholomew C Wacek, Elsevier (10 min)
      Reaching out to new user communities: the Metagenomics community
      Lynette Hirschman, The MITRE Corporation (10 min)
  • Open discussion with participants (25 min)
  • BioCreative Organizers: Cecilia N Arighi, Kevin B Cohen, Lynette Hirschman, Martin Krallinger, Zhiyong Lu, Alfonso Valencia, Thomas C. Wiegers, W John Wilbur, and Cathy H Wu

    References
    1.Hirschman, L., Yeh, A., Blaschke, C. and Valencia, A. (2005) Overview of BioCreAtIvE: critical assessment of information extraction for biology. BMC Bioinformatics, 6, S1.
    2.Krallinger, M., Morgan, A., Smith, L., Leitner, F., Tanabe, L., Wilbur, J., Hirschman, L. and Valencia, A. (2008) Evaluation of text-mining systems for biology: overview of the Second BioCreative community challenge. Genome Biology, 9, S1.
    3.Leitner, F., Mardis, S.A., Krallinger, M., Cesareni, G., Hirschman, L.A. and Valencia, A. (2010) An Overview of BioCreative II.5. IEEE/ACM Trans Comput Biol Bioinform., 7, 385-399.
    4.Arighi, C., Lu, Z., Krallinger, M., Cohen, K., Wilbur, W., Valencia, A., Hirschman, L. and Wu, C. (2011) Overview of the BioCreative III Workshop. BMC Bioinformatics, 12, S1.
    5.BioCreative IV Proceedings: http://www.biocreative.org/resources/publications/biocreative-iv-proceedings/
    6.Comeau D.C., Islamaj Doğan R., Ciccarese P., Cohen K.B., Krallinger M., Leitner F., Lu Z., Peng Y., Rinaldi F., Torii M., Valencia A., Verspoor K., Wiegers T.C., Wu C.H., Wilbur W.J. BioC: a minimalist approach to interoperability for biomedical text processing. Database (Oxford). 2013 Sep 18;2013:bat064.
    7.Arighi, C., Carterette B., Cohen, K.B., Krallinger, M., Wilbur, W., Fey, P., Dodson, R., Cooper, L., Van Slyke, C.E., Dahdul, W., Mabee, P., et al. (2013) An Overview of the BioCreative 2012 Workshop Track III: Interactive Text Mining Task. DATABASE, 2013:bas056.
    8.Arighi, C., Roberts, P., Agarwal, S., Bhattacharya, S., Cesareni, G., Chatr-aryamontri, A., Clematide, S., Gaudet, P., Giglio, M., Harrow, I. et al. (2011) BioCreative III interactive task: an overview. BMC Bioinformatics, 12, S4.
    9.Matis Mitchell S., Roberts P., Tudor C.O. and Arighi C.N. BioCreative IV Interactive Task. BioCreative IV Proceedings:Vol 1, pg190 (2013).

    Download ISB 2014 talks

    Downloads