Big Data Meets Privacy Regulation

May 31, 2018

European General Data Protection Regulation (GDPR) is just another step towards stricter regulation of how companies use collected data. It’s only a matter of time when similar provisions will appear in North America. As demonstrated by the recent high profile cases of privacy, Big Data needs to be collected, managed, and used responsibly. Moreover, the only practical way to ensure this, at scale, is by using aBig Data management system built on privacy and security at its core.

Big data is often viewed as an antithesis of privacy. Corporations, governments, and not for profits are amassing information about all their customers, prospects, and the general public including their opinions, their dreams and fears, locations, schedules, communications, social and business networks.

At its very best, that data is used to improve customer service, customize the user experience, and add value. Analytics-savvy organizations can automatically apply insights from data analytics to tailor product features that are a perfect fit for a customer’s needs.

However, sometimes our information is not sufficiently protected by the company. Even more alarming—all too often personal information is sold or otherwise shared without customer consent.

General Data Protection Regulation (GDPR), which was enacted in Europe on May 25th, introduces new data protection rules and company responsibilities. GDPR has an extraterritorial scope, and its transnational reach will likely affect the companies outside of the European Union as well. Also, if you follow the development of the recent high-profile privacy violation cases, it’s only a matter of time when similar privacy protection regulations will be implemented in North America and elsewhere.

Among other things, GDPR requires companies to designate the Data Protection officer role to undertake data protection impact assessment, implement risk-based technical and organizational measures to safeguard personal data and keep the record of data processing activities to prove compliance. For companies who were used to unconstrained access to personal data in their possession, this will prove to be a severe obstacle and may come with a high cost of implementation and noncompliance.

Our experience with data lake implementations shows that it is difficult to add privacy and security after the system has been deployed, and requires a combination of technology, people, and processes.

PHEMI was founded five years ago with the sole purpose of protecting individual privacy. We have a proprietary platform that was visionary in our perspective that Privacy by Design principles must provide the backbone of all corporate data and storage. We can help you design and implement a big data governance system and processes, provide training, and offer managed services if needed.

Our flagship product, PHEMI Central, allows data scientists and BI analysts to work with collected data while staying in compliance with privacy regulations and internal policies. At the same time, it provides privacy & compliance officers a unified view into what personal information is collected and stored, how it is de-identified and protected, what policies are defined for sharing and use, and audits how policies are being enforced.

Our flagship product, PHEMI Central, addresses big data governance needs of large enterprises in regulated industries.

Nowhere is privacy more critical than industries like healthcare and financial services, which is where PHEMI focuses our initial suite. PHEMI Central offers vertically integrated Hadoop-based data storage, flexible ingest, and processing, with a policy-based governance layer, wrapped into a user-friendly user interface. Compared with a conventional approach of building a custom governance layer based on open source tools like Ranger, Atlas, or Sentry, PHEMI Central provides a ready-to-use solution which shortens implementation time by months and offers attribute-based access control and risk-based de-identification.

Moreover, PHEMI Central offers managed services for everyone.

For companies who are only starting their Big Data journey or expanding its adoption beyond the first several use cases and do not have sufficient operational experience with distributed computing in Hadoop environment, PHEMI offers managed services.