Apple Acquires Dark Data Integrator Lattice.io

Apple revealed May 15 that it has acquired an artificial intelligence and machine learning startup called Lattice.io, which is able to take so-called dark data and turn it into usable information.

Financial information about the transaction was not released. News reports estimated the acquisition at between $175 million and $200 million.

Menlo Park, Calif.-based Lattice.io evolved from a Stanford University research project called DeepDive. The young company describes itself on Crunchbase as able to turn "dark data into structured data with human-caliber quality at machine-caliber scale."

Dark data is acquired through various computer network operations but not used in any manner to derive insights or for decision making. This is data that exists without labels or context, so it's difficult to find and utilize. Much of the time it consists simply of extra copies of the data. The ability of an enterprise to collect data can exceed the throughput at which it can analyze the data. In some cases, the organization may not even be aware that the data is being collected.
 

In an industrial context, dark data can include information gathered by sensors and telematics. There is a lot of dark data in the world; IBM has estimated that roughly 90 percent of data generated by sensors and analog-to-digital conversions is never used.

Organizations retain dark data for a lot of reasons, and it has been estimated that most companies only analyze 1 percent of their data. Often it is stored for regulatory compliance and record keeping. Some organizations believe that dark data could be useful to them in the future, once they have acquired better analytic and business intelligence technology to process the information.

Because storage is relatively inexpensive, storing data is easy. However, storing and securing the data usually entails greater expenses (or even risk) than the potential return profit.

Lattice says its intellectual property uses machine learning to process dark data and puts those results into a database. It enable users to extract, integrate, and predict problems in a single system allowing them to quickly build end-to-end data systems.

Lattice.io was founded by Christopher Re, Michael Cafarella, Raphael Hoffmann, and Feng Niu. Re is a professor of computer science at Stanford, and Carafella is a professor of computer science at the University of Michigan. Cafarella was co-creator of the popular Hadoop open source project that stores and analyzes large amounts of data.