When you are audited to any of the NERC Reliability Standards, federal regulations require you to present the audit team with evidence of your compliance with the requirements of the standard being audited. In this article I will give you my advice about gathering and presenting quality evidence.

In addition to audits, other compliance monitoring actions, such as spot checks, self-certifications, mitigation plans, certifications, etc., require evidence. In each of these monitoring actions, the burden to demonstrate compliance is on you, the registered entity. It is in your interest that the evidence you gather and submit in response to a monitoring activity is clear and concise.

Most monitoring actions are guided by the Generally Accepted Government Auditing Standards (GAGAS). The electronic (PDF) version of GAGAS is free, and the current (2024) version is available at www.gao.gov/yellowbook. In the discussion that follows, numbers in square brackets refer to the 2024 GAGAS chapter and paragraph.

Reasonable assurance

A compliance monitoring team, such as an audit team, must obtain reasonable assurance that you, the registered entity under review, are complying with the applicable Reliability Standards [8.19]. The team does this by obtaining sufficient, appropriate evidence to support their findings [8.90].

Sufficient, appropriate evidence

Sufficiency is a measure of the quantity of evidence needed to support the audit findings [8.99]. Appropriateness is a measure of the quality of evidence [8.102]. These two characteristics are related: when evidence is of higher quality, a lesser quantity is needed to support the audit findings [8.101].

Appropriate evidence is relevant, valid, and reliable [8.91]:

• Relevant evidence is evidence that has a direct relationship with the Reliability Standard and requirement being audited.

• Valid evidence means the evidence is a meaningful measure of the tasks performed to achieve compliance with the subject being audited.

• Reliable evidence refers to the consistency of evidence in demonstrating compliance [8.102].

When the evidence submitted is not either sufficient or appropriate, additional corroborating (or “stacking”) evidence may be requested by your audit team [8.102c]. Note that large volumes of evidence do not compensate for evidence that is not relevant, valid, or reliable [8.102d].

Types of evidence

There are three general types of evidence: physical, documentary, and testimonial [8.104].

Physical

Physical evidence is obtained by direct observation by the audit team. This could be a walkthrough of a physical facility. It might also be a live demonstration of a real-time capability such as response to an alarm.

Testimonial

Testimonial evidence is generally only used at a compliance audit for attestations and is considered the weakest form of evidence. Usually, an attestation will be used to document a negative, such as having no Reportable Cyber Security Incidents. Your subject matter experts (SMEs) may be interviewed by the audit team to explain the documentary evidence, but they will not be delivering sworn testimony.

Documentary

Documentary evidence may be written policies, plans, processes, procedures, or other types of documents. It may also be evidence that a task has been carried out.

Quality documentary evidence may answer some or all of these questions:

Is the document credible?	This is an assessment of the overall appearance of the document. Think of this as an indicator of the first impression an auditor might receive upon seeing the document. Does the appearance of the document help establish that it is genuine? Does the document appear to be altered?
What task was performed?	Each document presented as evidence should help build a coherent, consistent picture of how you implement your compliance program, and how compliance advances your security posture. Be sure the task being documented is clearly stated.
Who performed the task?	If there are questions about the task, who should be contacted?
Why was the task performed?	What security or compliance need did the task support?
Which assets were affected by the task?	An asset could be a physical Facility (such as a substation), a computer (Cyber Asset), or something else (such as roles and responsibilities lists). The evidence should indicate who or what was affected.
When was the task performed?	Dates are critical in demonstrating compliance. In general, evidence without dates is weak evidence.
Who authorized the task?	In security, and also compliance, it’s best to have multiple sets of eyes on any changes. A system of authorization and verification (the review, below) will add substantial credibility to the compliance evidence.
What was the state of the asset before the task was performed?	One of the best methods of demonstrating that a task was performed is to show the state of the asset both before and after the task. Examples of this include: a) for a process or procedure document a detailed version history is usually sufficient, although the prior version of the document should also be available if needed; b) for a software version update you should keep dated evidence of the version numbers before and after the update; c) for Windows patches keep a dated list of the installed KB numbers before and after the patch installation.
What was the state of the asset after the task was performed?
Was the result of the task reviewed, and if so, when and by whom?	A strong compliance program will include monitoring the performance of the compliance tasks, and keeping records of this monitoring is a good practice.

Note that not all of these questions will be applicable in every circumstance, but you should consider which questions can be answered by the evidence you gather.

Let’s look at an example of both weak and quality documentary evidence. CIP-003-8 (Security Management Controls) Requirement R3 is one of the simplest requirements to document. Figure 1 shows weak documentation of the designation of a CIP Senior Manager (the names have been changed for confidentiality).

While Figure 1 appears to meet the minimum necessary documentation per the language of the requirement, “Each Responsible Entity shall identify a CIP Senior Manager by name and document any change within 30 calendar days of the change,” this is actually very weak evidence. The CIP Senior Manager is identified by name and documented, but the evidence in Figure 1 will cause the audit team to have additional questions, such as: When was Jane designated as the CIP Senior Manager? Is there more than one Jane Smith working for the organization, and if so, which one is designated? Is there more than one NERC registered entity that this designation could apply to and if so, which does it apply to? The audit team will need to request corroborating (or “stacking”) evidence to achieve reasonable assurance that your CIP Senior Manager has been identified, and when this identification occurred.

Here’s what Figure 1 provides in terms of the quality questions above:

• Is the document credible? No, the designation is on plain paper and is unsigned
• What task was performed? Designation of CIP Senior Manager
• Who performed the task? Unknown
• Why was the task performed? Compliance with CIP-003-8 R3 (implied)
• Which assets were affected by the task? Unknown
• When was the task performed? Unknown
• Who authorized the task? Unknown
• What was the state of the asset before/after the task was performed? N/A
• Was the result of the task reviewed, and if so, by whom? Unknown

Figure 2, below, shows an example of quality evidence.

Here’s what Figure 2 provides in terms of the quality questions above:

• Is the document credible? Yes, the designation is on company letterhead and is signed by a senior management official
• What task was performed? Designation of CIP Senior Manager
• Who performed the task? R.C. Lincoln, CEO
• Why was the task performed? Compliance with CIP-003-8 R3 (implied)
• Which assets were affected by the task? NCR98765
• When was the task performed? 9/19/2023
• Who authorized the task? R.C. Lincoln, CEO
• What was the state of the asset before/after the task was performed? N/A
• Was the result of the task reviewed, and if so, by whom? Not applicable, per CEO authority

This evidence more clearly states the intention of the organization in designating its CIP Senior Manager.

Here’s some advice for keeping consistent quality for various types of documents you might use as evidence:

Document	Evidence
Policies, plans, processes, and procedures	Develop a standard template for this type of document that includes the document title, document revision number and date, company name or logo, applicable NCR numbers if you have multiple registrations, the person or group responsible for the document content, and the person, group or asset type the document applies to. Keep detailed revision histories and effective dates for each version of the document. For cyber security policies per CIP-003, make sure the CIP Senior Manager’s approval of each policy is clear and accompanied by the date of approval.
Periodic actions, such as asset list reviews	Dated documentation of actions performed is particularly important for periodic requirements. Ensure your documents clearly identify the action taken and the resulting changes, if any.
Event-triggered actions, such as incident response	Some of the CIP Standards require action based on a triggering event, such as incident response to a Cyber Security Incident, or a malware check for a Transient Cyber Asset. For these types of actions, I recommend that you use a checklist to ensure each required step of the applicable process or plan is performed. The checklist, when appropriately completed, signed, and dated, becomes your evidence of implementing the steps in your process or plan.
Ongoing actions, such as access control	For ongoing actions, such as managing a firewall ruleset to control electronic access to a Bulk Electric System asset, I suggest using a combination of change control and periodic review. The change control process will ensure that access control is not compromised by an unauthorized change, and the completed change control tickets will document this ongoing effort. The periodic review will ensure that old information is purged and current information is adequately documented.
General considerations	All documentary evidence should identify your company, should carry a date, and should be page numbered to show that the document is complete, and no pages are missing.

Evidence retention

The Compliance section of each Reliability Standard discusses the requirements for evidence retention for that standard. In some cases, a requirement may have different retention parameters for a certain type of evidence. An example of this might be the requirement to keep access logs for a period of 90 days. In this case, you should record the fact that you kept appropriate logs and that you deleted log records in compliance with the retention period in the requirement. One way to show this is by recording how many log records were kept for a particular day, and the date they were deleted.

Conclusion

This is a high-level overview of evidence considerations. There is much more to explore. I suggest you read the applicable sections of GAGAS, particularly Chapter 8 on performance audits. You may also want to read my article on How to Excel at a CIP Audit.

I’ve also gathered the recommendations above into a “Cheat Sheet,” available here.