The Complete Compliance Guide to Archiving of Online Data

Simply put, archiving is the law. Websites, email, enterprise collaboration platforms, social media accounts, and even mobile text messaging offer businesses the benefits of collaboration, flexibility, extended reach, and ease of communication, but they are also considered official business records by many industry regulators, which is why organizations are required to archive them on a regular basis.

For example, both the Financial Industry Regulatory Authority (FINRA) and the Securities and Exchange Commission (SEC) have put strict communication and recordkeeping regulations in place that encompass every form of data and communications that investment firms and other financial institutions engage in, from the recording of investment activities, to how client data should be kept safe, to the acceptable use of social media.

To give you some context, FINRA and the SEC view an official website as a company’s single biggest and most important point of contact between them and potential customers. It therefore follows that any information related through that website must comply with regulatory communication rules and guidelines. This covers the entirety of a business’s website content, including all updates, edits, amendments, and deletions made to the site.

SEC rules also make it clear that you can’t archive these documents in just any format that you consider most convenient. Archived data, including emails, chats on internal collaboration tools and internal social media conversations, must meet particular criteria, specifically that records must be easily accessible and that they must be timestamped in order to be ready for an SEC audit.

In addition, SEC Rule 17a-4(b) and FINRA Regulatory Notices 10-06 and 1139 include the following record-keeping guidelines:

• All marketing information, including websites, blogs and social media must be archived;
• Data must be archived in its original format (a screenshot or PDF is not the original format);
• Data must be retained for at least three years (depending on the type of content, website data must be archived for six years);
• Data must be stored in WORM format (write once, read many), backed up and indexed.
• Government agencies, on the other hand, or subject to the Freedom of Information Act (FOIA), Federal Records Act and Open Records Laws, which state that:
  • Both webpages and metadata must be collected;
  • Websites must be retained in their original format (HTML or WARC, not screenshots and PDFs).

All industries are subject to the Federal Rules of Civil Procedure (FRCP) Rule 34(b)(1)(c) and Federal Rule of Evidence – FRE rule 901(a):

• Obtain a screenshot of the web page as a visual reference;
• Obtain the source code of the web page in HTML or WARC format for digital forensics analysis;
• Gather the web server metadata (HTTP headers) for digital forensics analysis. This provides details on the web server collection date, time, IP address, web browser used etc;
• Place a SHA-256 bit digital signature and timestamp using an official digital certificate on all collected data to authenticate the collected evidence;
• Prepare all metadata in EDRM-XML for import in eDiscovery workflow tools.

What this all means in practice is that if, for instance, a records request is submitted for a social media post or online conversation, an organization has to be able to deliver this content.

This may sound simple, but a request could easily include thousands of posts, or dozens of comments that were deleted months ago, at which point a records request can suddenly become problematic and time-consuming.

As we know, organizations across industries are creating data at an unprecedented rate. Big data is vital to how we do business, deliver tailored customer experiences, and make better, smarter decisions. Unfortunately, too much data can be debilitating, and without effective recordkeeping measures in place, businesses can easily become non-compliant.

In fact, in an article published by Harvard Business Review, Leandro DalleMule, Chief Data Officer at AIG, and Thomas H. Davenport, author and Senior Adviser at Deloitte Analytics, point out that less than 1% of a typical organization’s unstructured data is analyzed or used at all, while “more than 70% of employees have access to data they should not, and 80% of analysts’ time is spent simply discovering and preparing data. Data breaches are common, rogue data sets propagate in silos, and companies’ data technology often isn’t up to the demands put on it.”

In other words, data is being collected, but it’s not being adequately leveraged, it’s often a huge compliance risk because the correct data permissions are not in place, and it isn’t stored or archived correctly.

What is Information Governance?

If we consider that the majority of large enterprises have a data problem, Information Governance becomes a critical component for any business.

Gartner defines Information Governance as “the specification of decision rights and an accountability framework to ensure appropriate behavior in the valuation, creation, storage, use, archiving and deletion of information. It includes the processes, roles, and policies, standards and metrics that ensure the effective and efficient use of information in enabling an organization to achieve its goals.”

While Information Governance isn’t a new concept, advanced technologies and competitive landscapes have changed how much data businesses are working with, as well as the complexities of complaint data recordkeeping.

According to the World Economic Forum, in 2019, on any given day, 500 million tweets were sent, 294 billion emails delivered to inboxes, 65 billion messages on WhatsApp, 4 petabytes of data were created on Facebook and 5 billion Internet searches were made. And those numbers are growing by the minute.

Regulatory requirements mean that organizations need to have control over any data that is generated by or in relation to their businesses, and even though this scale is far beyond any single company, the platforms that businesses use see massive amounts of data flow through them each day.

Old-school recordkeeping was reasonably straightforward. A single source of truth could easily be kept, and the organizational custodians of information were clear. This has been replaced with a new world where information governance professionals need to consider the impact of things like enterprise collaboration platforms, social media channels, application programming interfaces (APIs), and third-party integrations. In short, questions around where information resides, how it’s protected, who has access to it, and what companies should actually do with it are not as clear-cut as they once were.

When it comes to addressing the new challenges of information governance, the Association of Records Managers and Administrators (ARMA) International has developed eight principles that act as a standard of conduct. These illustrate how modern enterprises should:

• Oversee information management to ensure accountability within the organization;
• Manage information in a way that is open and transparent;
• Guarantee the authenticity and reliability of information;
• Classify and protect information that should not be accessible by all;
• Comply with all relevant recordkeeping regulations;
• Maintain the availability and accuracy of information;
• Retain information for regulatory, legal, and historical requirements;
• Dispose of information no longer required.

Archiving and Recordkeeping

With the wealth of information existing online, what’s published on a website or social media account now has the same legal status as a paper record. It therefore needs to be protected the same way—in a manner that is unalterable, and trustable. In regulatory terms, that means adhering to the FRE (Federal Rule of Evidence) and the FRCP (Federal Rules of Civil Procedure).

When organizations begin their Information Governance journeys, they progress through a number of levels before reaching Information Governance maturity. One of these levels can be classified as the ‘developing phase’, in which the business has realized the importance of information governance and is putting a governance program in place, but is still facing substantial risks because of a lack of systems and processes.

In terms of ensuring that data is unalterable and trustable, one of the biggest errors that we see at Pagefreezer during the ‘developing’ phase in organizations is confusion between backups and archiving.

An archive is a record. It is an account of a particular time that can be verified and authenticated. A backup, on the other hand, is a periodic snapshot of data that helps you recover records that get lost.

Most backups are saved for a few days or weeks until they make way for new backup data to overwrite it. This works well in the short-term in the case of an emergency recovery of data, but not in terms of recordkeeping, compliance and within an overall Information Governance framework.

This is because it’s relatively simple for anyone to take a screenshot, import it into Photoshop, and make changes. This is why the legal system does not accept backup versions as evidence – they’re just too easy to tamper with.

This is where archiving comes in, because it gives your organization ongoing access to business information as it once existed, for long periods of time.

Managing this data in the correct way and ensuring that the business is compliant at all times requires an in-depth information governance framework and automated archiving and recordkeeping tools.

Real-Time Archiving of Social Media Platforms

Many organizations already understand the importance of Information Governance when it comes to emails, but what of social media and collaboration platforms?

Communications on messaging apps and collaboration platforms are happening in real-time. So, while in some respects the recordkeeping requirements of a team collaboration platform or app-based messaging service are similar to that of email, in practical terms it’s quite different. On these channels, you’re essentially collecting and preserving real-time communications that are scattered across different platforms through various team channels, group chats, and private conversations.

Sure, emails can form long strings, but at the end of the day, each mail is self-contained and authored by a single person, and there are only single inboxes and outboxes that need to be archived.

At Pagefreezer, we’ve encountered several large financial institutions that have had to roll back a deployed enterprise collaboration platform, simply because the necessary compliance solutions were not in place. Taking a tool offline after everybody has already started relying on it for collaboration can have a serious impact on productivity and is a critical factor in how important Information Governance is in an organization’s larger data strategy.

Since the sudden shift to remote work caused by COVID-19, employees have become entirely dependent on platforms and solutions like Workplace from Meta, Microsoft Teams, Zoom, and G Suite to communicate and collaborate. As we’ve seen, this means that monitoring and managing this flow of data has become crucial as well.

Companies need to understand what data they’re holding and where it lives. The first crucial step to achieving this visibility is a comprehensive data inventory.

What Is a Data Inventory and Why Is It Important?

A data inventory (sometimes also called a data map) is a single source of truth—with relevant metadata—that provides instant insight into all sources of data a company has, what information these sources collect, where this data is stored, and what ultimately happens to it.

Metadata is the behind-the-scenes, back-end information that proves exactly when, where, and how a record was created. Without metadata, it’s very probable that digital evidence will be denied in court. There are four types of metadata that form the backbone of any data’s authenticity:

Client Metadata (who collected it): This includes browser, operating system, IP address, user.

Web Server/API Endpoint Metadata (where and when it was collected): URL, HTTP headers, type, date & time of request and response.

Account Metadata (who is the owner): Account owner, bio, description, location.

Message Metadata (what was said when): Author, message type, post date &  time, versions, links (un-shortened), location, privacy settings, likes, comments, friends.

Thanks to a data inventory, an organization can:

• Know which data leads to improved efficiency and increased accountability for everyone in the organization;
• Leverage data to achieve better overall reporting, decision-making and operational performance optimization;
• Assess any underlying risk and identify the controls an organization needs to protect its valuable information assets.

Most organizations only realise how much data they are collecting once they start taking a detailed inventory of all their data sources. Here are just some of the data sources that most businesses draw from each day:

• Accounting and point-of-sale software
• Customer relationship management (CRM) software
• Third-party applications and cloud-based solutions, like:
• Slack
• Salesforce
• Hubspot
•  Microsoft 365
• G Suite
• Electronic Data Interchange (EDI) software and solutions
• Websites with password-protected pages, forms, and chat bots
• Social media accounts that anyone can comment on or send a direct message to

The reality is that sensitive and valuable information is being collected across the organization every minute of every day. Each department collects and stores data for its own purposes, creating documents in Microsoft 365 and G Suite, and then sharing them through email and team collaboration tools. Without a full data inventory in place, it’s easy to overlook important data sources, but more importantly, given the regulatory environment most companies are dealing with these days, ignoring the problem is not an option.

Data inventory and the GDPR

Thanks to the prevalence of remote work and the rise of big data, information governance and proper data inventory is critical. However, thanks to the stringent requirements of modern privacy legislation, like the General Data Protection Regulation (GDPR) and California Consumer Privacy Act (CCPA), organizations are required by law to know exactly what user data they hold and what they do with it. Additionally, companies are expected to be able to respond effectively to a Data Subject Access Request (DSAR) or Right to Erasure Request.

The GDPR does not explicitly require a data inventory, but there’s no doubt that it makes compliance with these regulations much easier. Complying with the GDPR involves a number of key steps. These include:

• Understanding the definition of personal data under the GDPR
• Identify what personal data the business collects and how it is used
• Knowing where data is stored, including on any third-party systems
• Mapping the travel of data through the organization from the very first moment of collection (where data inventory comes in)
• Knowing exactly how long data is retained
• Understanding what the data looks like (is it structured or unstructured)

The Challenge of Creating a Data Inventory
The ultimate goal of a data inventory is simple in theory but can be challenging in practice. For decades, questionnaires, interviews, and Excel spreadsheets were enough. While these methods could theoretically still work today, it’s an extremely manual and labor-intensive process. Instead, a dedicated, automated solution is much better at streamlining the process. Given the risks that come with non-compliance, any solution that improves the data mapping process is well worth the investment.

There are four key steps that anyone responsible for legal compliance in an organization should consider:

1. Capture: You need to ensure that all new content is in some way being captured in case you eventually need it. This could be through a crawler on your website or through an API on your social media and team collaboration platforms.

2. Search: Since archived records can be overwhelming, you need a way to effectively search your archives so that you can actually find what you need, when you need it. For example, in the case of Pagefreezer, you can use the dashboard to search all of your archived websites and web pages for keywords. You can also search Workplace for specific users or keywords across all timelines, groups, channels, direct conversations, etc.

3. Collect: Simply searching your archives is not enough. You also need to be able to collect the records you’ve identified as relevant. Pagefreezer has a case management tool that allows you to add relevant records to a specific case folder to ensure that everything is kept together. You can also leave notes for yourself, or other reviewers, as you collect and add each record.

4. Export: Once you have what you need for an audit or legal matter, you need to be able to export it in a way that (i) shows the content is authentic and reliable and (ii) is in a format that is practical. In other words, something that can actually be submitted to an auditor or court of law. Generally speaking, this would be a PDF. Pagefreezer allows you to export records in PDF, complete with a digital signature and timestamp that proves authenticity.

The Multifaceted Nature of Data

All data falls into one of two categories: structured data and unstructured data. Structured data is anything that has a high consistency in terms of fields and values across database entries. This makes the relational nature of the data easy to understand and the database is quick to search. A bank account or employee directory are good examples of this.

Unstructured data, on the other hand, is literally everything else. It’s PDFs, text documents, spreadsheets, presentations, images, videos, and audio files. It’s emails and responses to emails, chats in collaboration tools and comments on documents. It’s texts and social media posts and comments and likes and shares. Basically, all the communication tools that modern enterprises use exist within the realm of unstructured data.

Deep-Linked Content

No data exists in isolation. In fact, the very nature of social media, websites, media sites and collaboration tools ensure that content is deeply connected across different platforms.

It’s therefore important to understand which types of content organizations are responsible for – and which types they are not. In the world of data, this means we need to review data entanglement and data adoption. Data entanglement occurs when two data stores share common information and a change in one is reflected in the other. Data adoption takes place when your business essentially takes ownership of data, even if it wasn’t generated by your organization to begin with.

For example, in the financial services sector, an unsolicited customer testimonial doesn’t meet FINRA criteria for entanglement or adoption. As a result, you may not need to archive it (although, we’d suggest, you might still want to). FINRA’s view of an unsolicited review can change however, if you “adopt” it by liking or sharing the post, which is seen as your organization giving the content your “seal of approval.” All adopted content must be archived, which means the second you’ve hit that like button, you now need a verifiable and authenticable archive of the post.

Along similar lines, if you hire an influencer to post an endorsement, that communication is “entangled.” Why? It’s reasonable to assume you have some impact over what an influencer posts, since they are being paid by your firm.

Hyperlinks pose another sticky problem for recordkeeping compliance. If a financial services firm links to third-party content, FINRA again considers this content “adopted” – and therefore the content should be archived, whether it belongs to the firm or not.

Evolving Platforms

The reality is that social media platforms are always evolving, which is why it’s important for an automated recordkeeping solution to be able to adapt to platform changes. If it can’t, every platform change will result in lengthy downtimes and record gaps. Similarly, if you’re relying on a manual process like screenshotting, you’re depending on a process that is labor-intensive, susceptible to data loss, and doesn’t result in records that’ll stand up in court. Instead, we would recommend some form of recordkeeping that collects data from various data sources automatically.

Real-Time Activity

Emails and PDFs are reasonably easy to archive, provided you have a data inventory in place based on a firm information governance policy. However, as we’ve just seen, social media content is always evolving. Not only can users chat and share files in real-time, but they can also edit their posts and delete content. This means that as soon as a record is created (by, for example, taking a screenshot), it is already outdated because the content has been altered. It also means that thousands of comments, likes, and shares can take place in an hour, and with each new interaction, a new record is required. In other words, a single post with lots of engagement can result in the creation of thousands of records in a very short space of time. This never-ending real-time activity poses a tremendous challenge, since a record can be outdated almost the moment that it’s created.

Capturing Content Before It Is Deleted

In a world of instant communications and real-time collaboration, it’s just as easy to post content as it is to delete it – and, as we know, particularly incriminating pieces of content are most likely to be edited or deleted by the user before they can be collected and preserved. It’s therefore important to capture content before it’s deleted, which cannot be achieved with a simple screenshot:

• Screenshotting a single tweet, Facebook post or Instagram post is quick and easy, but capturing an entire account this way can not only be time-consuming and frustrating, but prone to errors.
• There’s no way to prove the authenticity of a screenshot, which means that it does not meet the Federal Rules of Evidence, and in many cases will not satisfy an auditor or court. When it comes to presenting a screenshot in court, a sponsoring witness will need to prove that the image is real, and even then, opposing counsel will most likely question authenticity.
• A post can still be deleted after you’ve taken a screenshot of it, which means that you have an image (which, theoretically, could have been created or altered in Photoshop) and zero metadata, which makes it difficult to prove that your version is authentic.

Creating Authenticated Records

Regardless of whether records are needed for litigation or regulatory compliance, it is crucial to be able to prove each record’s authenticity. To prove the authenticity of digital records, organizations should not only collect data in original file formats, but also all associated metadata. Finally, the record should have a timestamp and digital signature at the time it was created. These elements are the only way that an organization can prove that the record is an accurate reflection of what appeared on a website, collaboration platform or social media account on a particular day.

How to Archive a Website

An official website is most likely your single biggest and most important point of contact between your company and potential customers. As we’ve seen, this means that you have a regulatory requirement to comply with the communication rules and guidelines provided by any regulators that pertain to your industry.

There are several ways to archive a website. A single webpage can simply be saved to your hard drive, free online archive tools such as HTTrack and the Wayback Machine can be used, or you can depend on a Content Management System (CMS) backup. These tools can be hard to work with and you’ll struggle to provide defensible evidence that can be submitted to an auditor or court, however. The best way to capture a site is therefore to use an automated archiving solution that captures every change – particularly if you have tens or even hundreds of pages on your site.

An automated website archiving service like Pagefreezer allows organizations to keep a complete record of website content. We use technology, similar to that used by search engines like Google, to crawl a site at regular intervals and capture all changes and deletions. Through our user-friendly dashboard, customers can then view chronological versions of any given page and instantly see what’s changed—deletions are highlighted in red and additions are shown in green.

With automated website capture software, finding what you’re looking for and exporting that data is also much easier than with a CMS backup. Pagefreezer offers advanced search that allows you to quickly find a specific keyword or phrase in an archive, and then export that information (complete with metadata) in PDF or WARC.

Social Media and Archiving

Late in 2018, the Securities and Exchange Commission charged two robo-advisers with false disclosures. Not only was this the first time that the SEC went after these modern software-driven investment operations, but it became a perfect example of how important compliant social media use and recordkeeping is.

The SEC argued that one company improperly retweeted client testimonials, while the other posted misleading information on social media and “failed to maintain required documentation.”

What does compliance look like though? Capturing and storing social media data isn’t as easy as simply archiving emails. Social media pages and accounts see constant activity with loads of posts, likes, unlikes, shares, and comments, both on display and hidden or deleted because of inappropriate language.

Archiving social media platforms is therefore complex and fraught with technical difficulties – but can be made simple with a tool like Pagefreezer.

How to Archive Twitter

Like Facebook and Instagram, Twitter allows users to download account data from the platform, which effectively creates an archive of the information that is essentially a local version of your online account. Unfortunately, as user-friendly as this view of the archive is, it has several severe limitations:

• It doesn’t contain all your data.
• Authenticity isn’t obvious.
• There’s no way to export data.

In practical terms, this local version isn’t all that different from the live online version, except that it’s no longer in real time, which means it also doesn’t contain the metadata you need to authenticate it.

Archiving Twitter for compliance and litigation

When it comes to collecting and preserving data for legal and compliance reasons, a much better alternative is to make use of an automated archiving solution that archives Twitter content in real-time. With real-time social media capture, you can:

• Capture replies and view them in the context of the original tweet;
• Capture direct messages;
• See associated metadata;
• Access deleted tweets that are no longer visible online;
• Authenticate archived content through a hash value (digital signature);
• Easily export documents to PDF in defensible quality;
• Place an account on legal hold to ensure nothing crucial is deleted from the archive.

How to Archive Facebook

Facebook Pages and Facebook Messages are considered to be business records. It’s therefore just as important to keep accurate records of who said what and when they said it on Facebook, as it is to keep any other corporate records.

The problem is that you cannot rely on Facebook to retain your content for you. In fact, the platform terms of use state that, “Facebook is not responsible for the actions, content, information, or data of third parties.” In addition, once posted, you’ve lost control of “your” content, as highlighted in Facebook’s terms of service, which state that, “you grant us a non-exclusive, transferable, sub-licensable, royalty-free, worldwide license to use any IP content that you post on or in connection with Facebook (IP License).”

Archiving Facebook for compliance and litigation

While you may want to debate the validity of all Facebook posts as business records, it’s best to anticipate audits and have proper social media policies and procedures in place beforehand. Start with a simple, easy-to-follow social media policy, teach all employees what it is, and then monitor content on an ongoing basis. From there, a proper archiving system that automatically handles the record-keeping aspect of your Facebook content will ensure you remain compliant.

With real-time Facebook capture, you can:

• Retain all context—archived content has the same look and feel as the original platform;
• Capture replies and view them in the context of the original post;
• Capture direct messages;
• See associated metadata;
• Access deleted posts or comments that are no longer visible online;
• Authenticate archived content through a hash value (digital signature);
• Easily export documents to PDF in defensible quality;
• Place an account on legal hold to ensure nothing crucial is deleted from the archive.

How to Archive Instagram

If you’re familiar with the platform, you’ll know that Instagram added an ‘Archive’ function in 2017, but the problem is that this isn’t really an archive feature at all. It’s actually more of a ‘hide’ function, which means it removes a post from your profile and hides it from other users without deleting it completely.

Instagram’s goal isn’t to capture and preserve a post in an archive that can be exported for use during an audit or legal matter. Instead, it’s just a quick and easy way for Instagram users to hide old pictures, either because they underperformed, or because they’re just too embarrassing to maintain.

Archiving Instagram for compliance and litigation

Since relying on Instagram’s own archiving capabilities isn’t an option when it comes to collecting and preserving data to meet recordkeeping requirements, many organizations resort to simple screenshotting of posts and comments.

This does retain the context of the information, but it also introduces its own issues:

• There is no way to prove the authenticity of a screenshot, which means that it does not meet the Federal Rules of Evidence and will not satisfy an auditor or court.
• Screenshotting a single Instagram post is quick and easy but capturing an entire account in this way can be very time-consuming, especially if new comments are constantly being posted.
• Because of the real-time nature of social media, it’s almost impossible to archive changes to posts and comments through screenshotting—a post might be edited multiple times or a comment might quickly be deleted before you’ve had time to screenshot it.

Instead, an automated archiving solution that collects and preserves Instagram content in real-time ensures that an organization can: 

• See all edited versions of a post.
• Retain all context—archived content has the exact same look and feel as the original platform.
• Instantly capture every new comment to every Instagram post across your account.
• See all metadata associated with a post.
• Authenticate archived content through a digital signature.
• Easily export documents to PDF.

Team Collaboration Tools and Archiving

Today, if your recordkeeping only covers emails, you’re missing half the story. Cloud-based collaboration platforms like Slack, Microsoft Teams and Trello allow users to direct message each other, create chat rooms, closed channels, share files, and edit documents in real-time. These communications are as relevant (and in many cases more so), than emails, which is why they should be treated in the same way as any other business communication.

Pagefreezer simplifies compliance and litigation by automatically archiving websites, social media, mobile text messages, and enterprise collaboration platforms through its archiving software solutions.

Our software solutions help organizations with the monitoring, capturing, and archiving of online data. By doing this, our tools help address critical business challenges, including Financial Industry Compliance, eDiscovery and Litigation Readiness, Monitoring and Data Loss Prevention, FOIA/Open Records Responses, GDPR & CCPA Compliance, Retention Scheduling and Legal Hold, Online Data Security and Information Governance of Online Data.

Visit one of our pages to learn more, and to explore our sector-specific solutions:

• Financial Sector Compliance
• Open Records Compliance and Response
• Information Governance of Online Data
• Pagefreezer for Financial Services
• Pagefreezer for Government
• Pagefreezer for Education
• Pagefreezer for Healthcare

• Archive Websites
• Monitor and Archive Social Media
• Archive Mobile Text Messages
• Enterprise Collaboration Monitoring and Capture
• Pagefreezer for Workplace

Alternatively, simply request a demo below to see Pagefreezer in action!