Concept Searching’s Award Winning conceptClassifier Solving the Big Data Challenge within the U.S. Department of Defense

Share Article

Filling the Big Data Gap with Solutions that Address Unstructured Content

Concept Searching’s technologies and framework analyze and extract highly correlated concepts from very large document collections. We are able to address the ‘Big Data’ challenge as it applies to unstructured content with highly scalable technologies.

Concept Searching, a global leader in semantic metadata generation, auto-classification, and taxonomy management software, and developer of the Smart Content Framework™, is now solving the challenge of extracting meaning from large repositories of unstructured content, filling the ‘Big Data’ gap with technology that addresses the problem of unstructured content. The extraction of meaning from content, scalability, and performance are the key components to address the issue. Currently working with many clients, including within the US Department of Defense, Concept Searching is successfully solving this problem.

Big Data deals with structured data, semi-structured or unstructured data, and unstructured content. One of the fundamental problems is the view that unstructured content must be managed in databases for analysis, in the same way as structured and semi-structured data is handled, which is not the best approach. Data is machine driven, whereas unstructured content is driven by people, which makes the nuances, insights, relationships of disparate content, sentiment, and knowledge capital much more difficult to extract.

From the Big Data view, turning everything into structured data is an option, but the current maturity of text analysis tools rate the certainty of the information at less than 70%. This is just data extraction, not concepts or ideas contained in the unstructured content. Unreliable information ultimately produces random garbage. A more accurate approach is needed.

Ensuring that the right information is available to end users and decision makers is fundamental to trusting the accuracy of the information. Once this has been accomplished, the content can be managed and used to extend the realm of unstructured content, beyond improving business processes such as search, records management, and data privacy. Organizations can then find the descriptive needles in the haystack to gain competitive advantage and increase business agility.

Concept Searching is currently solving the ‘Big Data’ problem for a US Department of Defense client, with petabytes of unstructured content. Using the Smart Content Framework™ it is approaching the management of unstructured content from a highly strategic as well as tactical implementation. By implementing the Concept Searching solution it will be solving records management, compliance, data privacy, and improving search outcomes with one set of technologies. From this framework it can then harness the power of its unstructured content assets to derive knowledge capital. Other clients, in particular a global financial investment analyst firm and another an international engineering, architecture and consulting firm, are utilizing the technologies to extract value from approaching 100 terabytes or more of unstructured content.

According to Martin Garland, President of Concept Searching, “Concept Searching’s technologies and framework analyze and extract highly correlated concepts from very large document collections. We are able to address the ‘Big Data’ challenge as it applies to unstructured content with highly scalable technologies. This enables our clients to attain an ecosystem of semantics that delivers understandable results.” He continued, “The valuable insight gained can be used to identify competitive advantages, customer perception, regional trends, and, perhaps more importantly, identify the internal knowledge capital that exists but is rarely used because it cannot be found.”

Going beyond a technology solution, Concept Searching’s Smart Content Framework™ for information governance is a platform independent toolset that provides the enterprise framework to mitigate risk, automate processes, manage information, protect privacy, and address compliance issues.

Underlying the Smart Content Framework™, the technologies provide the ability to transparently tag content, classify it to organizational taxonomies, preserve and protect information through the automatic identification of records and privacy data, and act as a migration tool. The result is one set of technologies that improves search, provides an enterprise metadata repository, automates records management, ensures compliance, protects and secures data, and facilitates migration.

Additional information can be found in the Concept Searching blog and in the Big Data Solution Overview. To keep up-to-date on pertinent knowledge management trends, sign up for Concept Searching’s newsletter ”The View”.

Follow Concept Searching:

About Concept Searching
Founded in 2002, Concept Searching provides software products that deliver conceptual metadata generation, auto-classification, and powerful taxonomy management from the desktop to the enterprise. Concept Searching, developer of the Smart Content Framework™, provides organizations with a method to mitigate risk, automate processes, manage information, protect privacy, and address compliance issues. This information governance framework utilizes a set of technologies that encompasses the entire portfolio of information assets, resulting in increased organizational performance and agility.

Concept Searching is the only platform independent statistical metadata generation and classification software company in the world that uses concept extraction and compound term processing to significantly improve access to unstructured information. The Concept Searching Microsoft suite of technologies runs natively in SharePoint 2010, FAST, Windows Server 2008 R2 FCI, and in Microsoft Office applications.

The building blocks of Concept Searching’s Smart Content Framework™ are being used by organizations from a diverse number of industries including the US Army, the US Air Force, the UK MOD, Baker Hughes, Deloitte, Logica, NASA Safety Center, OppenheimerFunds, Point B, Perkins+Will, Parsons Brinckerhoff, Burns & McDonnell,, the US Department of Health & Human Services, Transport for London, the London Fire Brigade, the National Transportation Safety Board, and Xerox.

Headquartered in the US with offices in the UK, South Africa and Canada, Concept Searching solves the problem of finding, organizing, and managing information capital far beyond search and retrieval. The technologies are being used to drive intelligent migration, enable effective records management, identify and secure sensitive information and enhance governance. For more information about Concept Searching’s solutions and technologies please visit

All product and company names herein may be trademarks of their respective owners.

Share article on social media or email:

View article via:

Pdf Print

Contact Author

Nicola Barnes
Visit website