Satellite Banner
Next Gen Sequencing
Scientific Community
 
Become a Member | Sign in
Home>News>This Article
  News
Return

What's the Big Deal about Big Data?

Published: Monday, September 17, 2012
Last Updated: Monday, September 17, 2012
Bookmark and Share
The data-intensive nature of scientific research is currently driving the emergence of big data solutions that can gather, analyze, and transport extremely large volumes of data among multiple locations worldwide.

Wikipedia defines big data as "a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools. The challenges include capture, storage, search, sharing, analysis, and visualization".

Laboratories have been dealing with large amounts of data for decades, with the volume increasing dramatically every year and the trend now being toward larger data sets.  The problem has long been how to manage and mine that data for relevant information.  In the current data-intensive environment, the difficulty executing data management tasks has increased exponentially.

What's interesting is how big data is changing the nature of data management in the lab.  Relational databases and desktop statistics and visualization packages that have been so effective previously are not up to the task.  Instead, big data utilizes massively parallel software running on a large number of servers, typically more than any one business can afford.

One such solution is an open-source NoSQL database that is designed for massive amount of data delivery over web and cloud applications. NoSQL databases do not use tables and thus generally do not use SQL as the query language. What they do use is a distributed, fault-tolerant architecture that manages the data redundantly on multiple servers.

NoSQL databases don't replace databases such as Oracle RDBMS, instead they provide an entirely new way to manage data because they allow applications to collect and analyze massive amounts of information from numerous sources.

Life sciences laboratories are particularly affected by the big data trend.  When it comes to genomics, for instance, petabyte-scale networks are emerging that better support genomic research and emerging clinical requirements.  

There is also growing demand to manage big data using cloud computing platforms, and to move large volumes of next-gen DNA sequencing and research data at high-speed over vast distances.  The challenges of performing these activities into and out of the cloud are being addressed. This area has been led by Genentech, one of the early adopters of big data and cloud computing solutions to support their research.

Perhaps laboratories should have seen this coming since it is the inevitable result of better instrumentation that generates more data faster that then needs better analytical solutions–but hindsight is always 20/20. 


Further Information

Join For Free

Access to this exclusive content is for Technology Networks Premium members only.

Join Technology Networks Premium for free access to:

  • Exclusive articles
  • Presentations from international conferences
  • Over 4,000+ scientific posters on ePosters
  • More than 5,300+ scientific videos on LabTube
  • 35 community eNewsletters


Sign In



Forgotten your details? Click Here
If you are not a member you can join here

*Please note: By logging into TechnologyNetworks.com you agree to accept the use of cookies. To find out more about the cookies we use and how to delete them, see our privacy policy.

Related Content

Big Genetics in BC: The American Society for Human Genetics 2016 Meeting
Themes at this year's meeting ranged from the verification, validation, and sharing of data, to the translation of laboratory findings into actionable clinical results.
Friday, November 11, 2016
Cancer Genetics: Key to Diagnosis, Therapy
When applied judiciously, cancer genetics directs caregivers to the right drug at the right time, while sparing patients of unnecessary or harmful treatments.
Tuesday, October 25, 2016
Integrated Omics Analysis
Studying multi-omics promises to give a more holistic picture of the organism and its place in its ecosystem, however despite the complexities involved those within the field are optimistic.
Wednesday, October 19, 2016
Know your Enemy
Genetic markers for mosquitos which prefer humans to cattle identified for improved malaria surveillance and control.
Wednesday, October 05, 2016
What Makes a Good Scientist?
It’s the journey, not just the destination that counts as a scientist when conducting research.
Wednesday, June 15, 2016
AGBT: Going Beyond Sequencing
Advances in Genome Biology and Technology 2016 saw a strong focus on the workflow: 'sample in, insight out'.
Thursday, February 25, 2016
Scientific News
Big Genetics in BC: The American Society for Human Genetics 2016 Meeting
Themes at this year's meeting ranged from the verification, validation, and sharing of data, to the translation of laboratory findings into actionable clinical results.
Accelerating the Detection of Foodborne Bacterial Outbreaks
The speed of diagnosis of foodborne bacterial outbreaks could be improved by a new technique developed by researchers at the Georgia Institute of Technology.
Antibiotics Discovered While Sifting Through Human Microbiome
Researchers identify genes in a microbe’s genome that produce antibiotic compounds, then synthesize them without the need for bacterial culture.
A Genome-wide View of Human DNA Viruses
In this study, Duplex sequencing was used to accurately analyse the genome-wide rate of spontaneous mutation of human adenovirus C5 (HAdv5).
Peer Review is in Crisis, But Should be Fixed, Not Abolished
After the time to get the science done, peer review has become the slowest step in the process of sharing studies, and some scientists have had enough.
Rare Immunodeficiency Yields Unexpected Insights
Scientists uncover previously unknown gene mutation revealing the role of a key molecule involved in immune cell development.
Cancer Genetics: Key to Diagnosis, Therapy
When applied judiciously, cancer genetics directs caregivers to the right drug at the right time, while sparing patients of unnecessary or harmful treatments.
Illumina Contributes to ClinVar Database
The contribution includes variants of all classifications, from pathogenic to benign, identified during interpretation of whole genome sequences generated in the CLIA-certified, CAP-accredited Illumina Clinical Services Laboratory.
Agilent Presents Early Career Professor Award to Dr. Roeland Verhaak
JAX professor recognized for the development and implementation of workflows for the analysis of big-data from transcriptomics to next generation sequencing approaches.
Integrated Omics Analysis
Studying multi-omics promises to give a more holistic picture of the organism and its place in its ecosystem, however despite the complexities involved those within the field are optimistic.
Skyscraper Banner

SELECTBIO Market Reports
Go to LabTube
Go to eposters
 
Access to the latest scientific news
Exclusive articles
Upload and share your posters on ePosters
Latest presentations and webinars
View a library of 1,800+ scientific and medical posters
4,000+ scientific and medical posters
A library of 2,500+ scientific videos on LabTube
5,300+ scientific videos
Close
Premium CrownJOIN TECHNOLOGY NETWORKS PREMIUM FOR FREE!