Search Knowledge Base Articles

Controls Troubleshooting & Root Cause Analysis

CE-SOP-008 — CONTROLS TROUBLESHOOTING & ROOT CAUSE ANALYSIS

Department: Engineering – Electrical Controls & Automation
Document Type: Standard Operating Procedure
Document ID: CE-SOP-008
Revision: Rev. 1.0
Status: Active Standard
Process Owner: Electrical Controls Engineering
Related Documents: CE-SOP-001, CE-SOP-002, CE-SOP-004, CE-SOP-006, CE-SOP-007

1. PURPOSE

The purpose of this Standard Operating Procedure is to establish a safe, systematic, evidence-based method for troubleshooting electrical controls and automation systems.

The objective is to identify:

WHAT FAILED

WHERE IT FAILED

WHY IT FAILED

HOW OPERATION CAN BE SAFELY RESTORED

and, where appropriate:

HOW RECURRENCE CAN BE PREVENTED

The troubleshooting process shall prioritize:

SAFETY → OBSERVATION → SIGNAL TRACING → ROOT CAUSE → CORRECTIVE ACTION → VERIFICATION → DOCUMENTATION

2. CORE TROUBLESHOOTING PRINCIPLE

DO NOT CHANGE THE PROGRAM SIMPLY BECAUSE THE MACHINE DOES NOT RUN.

The PLC may be correctly preventing operation because:

A permissive is missing
An interlock is active
A sensor is not made
Feedback is missing
A fault is active
Communication is lost
Equipment is in the wrong mode
A physical device has failed
A process condition is incorrect

Troubleshooting begins with understanding the failure—not bypassing it.

3. SCOPE

This SOP applies to troubleshooting involving:

PLCs
HMIs
SCADA
Digital inputs
Digital outputs
Analog signals
Sensors
Photo eyes
Proximity switches
Limit switches
Valves
Solenoids
Motors
Starters
VFDs
Servo systems
Remote I/O
Industrial networks
Ethernet/IP
PROFINET
Modbus
PLC-to-PLC communication
Alarms
Sequences
Interlocks
Permissives
Timers
Counters
Control panels
Power supplies
Field wiring

4. TROUBLESHOOTING PHILOSOPHY

Controls troubleshooting should answer four questions:

QUESTION 1 — WHAT SHOULD BE HAPPENING?

Understand expected operation.

QUESTION 2 — WHAT IS ACTUALLY HAPPENING?

Observe the real condition.

QUESTION 3 — WHERE DO EXPECTED AND ACTUAL BEHAVIOR SEPARATE?

Find the failure boundary.

QUESTION 4 — WHY?

Determine root cause.

This prevents random replacement of components or unnecessary program changes.

5. STANDARD TROUBLESHOOTING PATH

Use the following general workflow:

MAKE SAFE

↓

DEFINE THE SYMPTOM

↓

VERIFY OPERATING CONDITIONS

↓

CHECK FIELD DEVICE

↓

CHECK PHYSICAL INPUT

↓

CHECK PLC INPUT

↓

CHECK I/O MAPPING

↓

CHECK MODE

↓

CHECK PERMISSIVES

↓

CHECK INTERLOCKS

↓

CHECK SEQUENCE LOGIC

↓

CHECK COMMAND

↓

CHECK PLC OUTPUT

↓

CHECK OUTPUT MODULE

↓

CHECK FIELD WIRING

↓

CHECK PHYSICAL DEVICE

↓

CHECK FEEDBACK

↓

IDENTIFY ROOT CAUSE

↓

CORRECT

↓

TEST

↓

MONITOR

↓

DOCUMENT

6. STEP 1 — MAKE THE SYSTEM SAFE

Before troubleshooting:

Identify hazards
Determine equipment state
Determine whether motion may occur
Follow applicable energy-control procedures
Communicate with affected personnel
Confirm safe access

Troubleshooting does not override established electrical or machine-safety procedures.

7. STEP 2 — DEFINE THE SYMPTOM

Avoid vague statements.

Poor:

"Line 5 isn't working."

Better:

"Line 5 conveyor does not start in Auto."

Better still:

"Line 5 conveyor does not start in Auto after the Start command. HMI shows Auto selected and no visible alarm."

A precise symptom creates a better diagnostic starting point.

8. EXPECTED VS. ACTUAL

Document:

EXPECTED

What should happen?

ACTUAL

What happens instead?

Example:

Expected:
Can Stop retracts after jam is cleared and operator presses Reset.

Actual:
Can Stop remains active after photo eye clears and Reset is pressed.

The difference becomes the troubleshooting target.

9. STEP 3 — GATHER HISTORY

Ask:

When did the problem begin?
Did it ever work correctly?
Is it constant or intermittent?
What product was running?
What mode was active?
What happened immediately before failure?
Was maintenance performed?
Was hardware replaced?
Was a PLC/HMI change made?
Was power lost?
Did a network event occur?
Does it fail every cycle?
Does it affect one shift only?

One of the most valuable questions is:

WHAT CHANGED?

10. CHECK RECENT ENGINEERING ACTIVITY

Review:

Recent Change Logs
Recent Support Tickets
Recent PLC modifications
Recent HMI modifications
Hardware replacement
Firmware changes
Network changes
Electrical work

Recent changes do not automatically mean they caused the problem.

They are evidence worth reviewing.

11. STEP 4 — OBSERVE BEFORE TOUCHING

Observe the machine before changing anything where conditions allow.

Look for:

Alarm messages
HMI status
Stacklights
Sensor LEDs
I/O LEDs
Drive faults
PLC faults
Communication indicators
Mechanical obstruction
Valve position
Operator actions
Unusual sounds
Loose wiring
Damaged devices

Changing conditions too early may destroy useful evidence.

12. STEP 5 — VERIFY OPERATING MODE

Determine:

Auto?
Manual?
Maintenance?
Setup?
Disabled?
Bypass?

Many apparent failures are mode-related.

Verify the PLC's internal mode—not only what the HMI appears to display.

13. STEP 6 — CHECK ACTIVE ALARMS

Review:

Active alarms
Alarm history
First-out alarm where available
Device faults
Drive faults
Communication alarms
Safety faults

Do not repeatedly reset alarms without first understanding why they occurred.

14. STEP 7 — START AT THE PHYSICAL CONDITION

For a missing input ask:

IS THE PHYSICAL CONDITION ACTUALLY TRUE?

Example:

PLC expects:

Infeed_PE = ON

Verify:

Is product physically present?
Is sensor aligned?
Does sensor LED activate?
Is sensor powered?
Is reflector clean?
Is target detectable?

Do not start by blaming the PLC.

15. INPUT SIGNAL TRACE

Use:

PROCESS CONDITION

↓

SENSOR

↓

SENSOR LED

↓

FIELD WIRING

↓

TERMINAL

↓

INPUT MODULE

↓

PHYSICAL PLC ADDRESS

↓

I/O MAPPING

↓

INTERNAL TAG

↓

CONTROL LOGIC

↓

HMI STATUS

Find where the signal stops matching reality.

16. INPUT MODULE TEST

If sensor activates but PLC does not see it:

Check:

Input module LED
Correct channel
Voltage
Common
Wiring
Terminal
Module health
Physical address
I/O mapping

This identifies whether the failure is:

FIELD SIDE

PROGRAM SIDE

17. I/O MAPPING CHECK

Verify:

Correct physical address
Correct internal tag
Correct mapping instruction
Correct routine execution
No duplicate or incorrect mapping

Example:

Physical input may work correctly while internal tag remains false because the mapping is wrong.

18. OUTPUT SIGNAL TRACE

For an output that does not operate:

OPERATOR / SEQUENCE REQUEST

↓

MODE

↓

PERMISSIVES

↓

INTERLOCKS

↓

COMMAND LOGIC

↓

OUTPUT MAPPING

↓

PHYSICAL OUTPUT

↓

OUTPUT MODULE

↓

FIELD WIRING

↓

DEVICE

↓

FEEDBACK

Trace the chain in order.

19. COMMAND VS. OUTPUT

Distinguish:

COMMAND

from

PHYSICAL OUTPUT

Example:

Can_Stop_CMD

may be TRUE while:

Local:2:O.Data.4

is FALSE.

That immediately narrows the problem.

20. OUTPUT MODULE TEST

If PLC output is commanded but device does not operate:

Check:

Output module LED
Output voltage
Fuse
Common
Interposing relay
Terminal
Wiring
Device coil
Air supply
Mechanical condition

Do not modify PLC logic when the PLC is already correctly commanding the output.

21. FEEDBACK TEST

If output operates physically but sequence does not advance:

Check feedback.

Examples:

Valve Open Feedback
Motor Running Feedback
Cylinder Extended
Drive At Speed
Pressure OK

The command may be working correctly while confirmation is missing.

22. PERMISSIVE TROUBLESHOOTING

When a machine will not start:

Find:

WHICH REQUIRED CONDITION IS FALSE?

Examples:

System Enabled
Drive Healthy
Air Pressure OK
Downstream Ready
Guard Closed
No Active Fault

Trace each permissive individually.

Where possible, HMI diagnostics should expose missing permissives.

23. INTERLOCK TROUBLESHOOTING

When an output unexpectedly stops:

Determine:

WHICH CONDITION IS PREVENTING OPERATION?

Examples:

Jam
Overload
High Temperature
Downstream Blocked
Communication Fault
Valve Conflict

Do not immediately bypass the interlock.

First determine why it is active.

24. SEQUENCE TROUBLESHOOTING

For sequential logic determine:

CURRENT STEP

EXPECTED NEXT STEP

CONDITION REQUIRED TO ADVANCE

Example:

Current:

Step_30_Fill

Step_40_Mix

Required:

Tank_Level_High = TRUE

If the system remains in Step 30, troubleshoot why the transition condition is missing.

25. TIMER TROUBLESHOOTING

Check:

Enable condition
Accumulator
Preset
Done bit
Reset condition
Retentive behavior

Common timer issues:

Enable condition flickers
Timer constantly resets
Wrong time base
Incorrect preset
HMI value overwrites preset
Done bit used incorrectly

26. COUNTER TROUBLESHOOTING

Check:

Count event
One-shot/event behavior
Accumulator
Preset
Reset
Retention

Watch for repeated counting while an input remains continuously true.

27. ANALOG SIGNAL TROUBLESHOOTING

Trace:

PROCESS VALUE

↓

SENSOR / TRANSMITTER

↓

ELECTRICAL SIGNAL

↓

ANALOG MODULE

↓

RAW VALUE

↓

SCALING

↓

ENGINEERING VALUE

↓

PLC LOGIC

↓

HMI DISPLAY

Check:

4–20 mA
0–10 V
Raw counts
Scaling
Units
Broken wire behavior
Calibration

28. HMI TROUBLESHOOTING

If PLC logic appears correct but HMI does not:

Check:

HMI communication
Shortcut/path
Tag reference
Display connection
Button configuration
Security
Animation
Numeric entry
Runtime file
Correct application

Determine whether problem exists in:

PLC

HMI

before modifying either.

29. HMI BUTTON TROUBLESHOOTING

For a button that does not work verify:

Correct tag
Button type
Momentary behavior
Set On
Set Off
Toggle
Write value
Security
PLC receives command
PLC logic accepts command

The HMI may write correctly while PLC logic rejects the command because a permissive is missing.

30. NETWORK TROUBLESHOOTING

For communication failures check:

Device power
Physical cable
Link LEDs
IP address
Subnet
Duplicate IP
Switch port
Device status
PLC connection
Protocol configuration
Managed switch diagnostics
Network path

Troubleshoot from physical layer upward.

31. VFD TROUBLESHOOTING

Check:

Drive power
Fault code
Ready status
Start command
Speed reference
Communication
Motor overload
Interlock
Feedback
Local/Remote mode

Record fault codes before resetting where practical.

The fault code may be the best evidence available.

32. VALVE TROUBLESHOOTING

Trace:

PLC Command

↓

PLC Output

↓

Output Module

↓

Solenoid Voltage

↓

Solenoid

↓

Air/Process Supply

↓

Valve Movement

↓

Feedback

Possible causes include:

No PLC command
Interlock
Failed output
Failed solenoid
No air
Mechanical valve failure
Missing feedback

33. MOTOR TROUBLESHOOTING

Trace:

Start request
Permissives
Interlocks
PLC output
Starter/VFD command
Overload
Power
Motor
Running feedback

Do not assume every motor-not-running condition is a PLC issue.

34. FIRST-OUT INFORMATION

Where multiple faults occur together, identify:

WHICH CONDITION OCCURRED FIRST?

The first event often reveals root cause.

Example:

Drive Communication Fault
Motor Not Running
Downstream Not Ready
Line Stopped

The drive communication fault may be the initiating event.

35. EVENT TIMELINE

For complex incidents build a timeline.

Example:

10:32:14 – Network communication lost.

10:32:16 – VFD status unavailable.

10:32:18 – Conveyor fault generated.

10:32:19 – Line stopped.

Timeline analysis helps separate cause from consequence.

36. TEMPORARY DIAGNOSTIC LOGIC

Diagnostic logic may be added when necessary.

Examples:

Event counters
First-out bits
Timestamp capture
Temporary trend tags
Diagnostic status bits

Temporary diagnostic logic shall follow:

CE-SOP-004 – Controls Change Management

It shall be removed or intentionally retained/documented afterward.

37. TEST BITS

Test bits may be used when necessary.

Use descriptive names.

Example:

TEST_AV19_Open

not:

B3:0/7 Test

where descriptive naming is supported.

Test bits should not remain active unintentionally.

38. FORCING INPUTS / OUTPUTS

Forces may assist troubleshooting but introduce risk.

Before forcing:

☐ Understand physical consequence

☐ Verify correct I/O

☐ Establish safe equipment state

☐ Notify affected personnel where required

After testing:

☐ Remove force

☐ Verify force removed

☐ Verify normal logic restored

☐ Document significant use

39. FORCE RULE

A FORCE IS A DIAGNOSTIC TOOL — NOT A PERMANENT REPAIR.

Never leave an unintended force active.

40. BYPASS DURING TROUBLESHOOTING

A temporary bypass may sometimes be necessary for controlled testing.

Before bypassing determine:

What is being bypassed?
Why?
What protection is lost?
Is it process or safety related?
Who authorizes it?
How will it be indicated?
When will it be removed?

Follow CE-SOP-004.

41. PROGRAM MODIFICATION DURING TROUBLESHOOTING

If troubleshooting reveals the PLC/HMI program requires modification:

Stop treating the activity as pure troubleshooting.

Create or update:

Support Ticket
Change Record
PRECHANGE Backup
Test Plan

Then follow:

CE-SOP-004 – Controls Change Management

and

CE-SOP-007 – Controls Commissioning

42. TROUBLESHOOTING VS. ENGINEERING CHANGE

Example:

Finding: Sensor bracket is loose.

Action:

Repair bracket.

No PLC change required.

Example:

Finding: Existing PLC logic has no timeout for failed valve feedback.

Action:

This is now an engineering change.

Create:

Controls Change / Project

Do not casually add logic during troubleshooting without controlling the change.

43. ROOT CAUSE

Root cause is the underlying condition that caused the failure.

Example:

Symptom:

Conveyor stopped.

Immediate cause:

Motor overload tripped.

Root cause:

Bearing failure increased mechanical load.

Replacing or resetting the overload without addressing the bearing may restore operation temporarily but does not resolve root cause.

44. ROOT CAUSE CATEGORIES

Recommended categories:

PLC Logic
HMI Configuration
Sensor Failure
Output Device
Field Wiring
Electrical Power
Power Supply
Network
Communication
VFD
Servo
Mechanical
Pneumatic
Hydraulic
Process
Operator
Configuration
Firmware
Software
Hardware Failure
Environmental
Documentation
Design Deficiency
Unknown
Other

45. FIVE WHYS

For recurring or significant problems, use the Five Whys where useful.

Example:

Why did Line 5 stop?
Can Stop activated.

Why?
Jam timer expired.

Why?
Photo eye remained blocked.

Why?
Product accumulated.

Why?
Downstream conveyor stopped unexpectedly.

The real problem may not be the Can Stop logic.

The logic may have worked correctly.

46. ROOT CAUSE VS. CONTRIBUTING FACTOR

Distinguish:

ROOT CAUSE

Primary underlying reason.

CONTRIBUTING FACTOR

Condition that increased likelihood or severity.

Example:

Root Cause: Failed sensor cable.

Contributing Factor: Cable routed where repeated mechanical contact occurred.

Both may require corrective action.

47. CORRECTIVE ACTION LEVELS

LEVEL 1 — RESTORE

Get equipment safely operating.

LEVEL 2 — REPAIR

Correct failed component/condition.

LEVEL 3 — PREVENT

Address root cause to reduce recurrence.

LEVEL 4 — IMPROVE

Improve diagnostics/design so future failures are easier to identify.

Strong Controls Engineering aims beyond Level 1 whenever practical.

48. TEMPORARY CORRECTIVE ACTION

Document:

What was done
Why temporary
Risk
Owner
Permanent solution
Target date

Example:

Temporary: Repositioned photo eye.

Permanent: Replace damaged mounting bracket during scheduled downtime.

49. PERMANENT CORRECTIVE ACTION

A permanent corrective action should address the root cause where practical.

Examples:

Replace failed sensor
Repair wiring
Replace damaged connector
Improve cable routing
Correct logic defect
Add missing diagnostic alarm
Replace obsolete hardware
Modify mechanical design
Improve preventive maintenance

50. VERIFICATION AFTER REPAIR

After corrective action verify:

DID THE ORIGINAL FAILURE ACTUALLY DISAPPEAR?

Test:

Normal operation
Original failure condition where appropriate
Reset
Recovery
Multiple cycles

Do not close based solely on:

"It started."

51. REPEAT FAILURE RULE

If the same or substantially similar failure repeatedly occurs:

STOP TREATING EACH EVENT AS AN ISOLATED INCIDENT.

Initiate:

RECURRING FAILURE REVIEW

Consider:

Root Cause Analysis
Engineering Project
Hardware upgrade
Additional diagnostics
Alarm improvement
Mechanical redesign
Preventive maintenance
Operator training
Vendor involvement

52. RECURRING FAILURE THRESHOLD

Your web system may automatically flag:

REPEAT FAILURE

when the same asset/root-cause category exceeds a defined threshold.

Example:

3 similar Support Tickets within 30 days

The exact threshold may be adjusted based on equipment criticality.

53. BAD ACTOR EQUIPMENT

Track assets generating disproportionate Controls Engineering support.

Dashboard example:

TOP CONTROLS BAD ACTORS

Line 5 – 14 tickets
RO Skid – 11 tickets
Line 3 Sleever – 8 tickets

This converts troubleshooting history into reliability strategy.

54. ROOT CAUSE TRENDING

Track failures by category.

Example:

Sensors: 32%

Wiring: 21%

Network: 15%

PLC Logic: 9%

Drives: 8%

This may reveal where engineering resources should be invested.

55. TROUBLESHOOTING ACTIVITY LOG

For significant troubleshooting record:

Date/Time

Action

Observation

Conclusion

Next Step

Example:

09:14
Checked physical infeed PE. LED changes normally.

09:18
PLC physical input changes normally.

09:21
Internal mapped tag remains OFF.

09:24
Found incorrect I/O mapping.

This creates a useful diagnostic history.

56. EVIDENCE COLLECTION

Useful evidence includes:

PLC screenshots
HMI screenshots
Alarm history
Trend data
Drive fault history
Photos
Voltage measurements
Network diagnostics
Event logs
Operator statements

Capture evidence before resetting or power-cycling where practical.

57. DON'T RESET THE EVIDENCE AWAY

Repeatedly pressing Reset can erase useful information.

Before resetting consider recording:

Alarm
Fault code
PLC state
Drive state
Sequence step
Timer accumulator
Input state
Output state

Then reset when appropriate.

58. INTERMITTENT PROBLEMS

For intermittent faults consider adding:

Event counters
Trends
First-out logic
Timestamps
Alarm history
Diagnostic bits

The objective is to capture the system state when the fault occurs.

59. TRENDING

Useful signals may include:

Sensor state
Command
Feedback
Sequence step
Timer
Alarm
Speed
Pressure
Temperature
Communication status

Trend:

CAUSE + RESPONSE

not simply the final alarm.

60. ESCALATION

Escalate when:

Safety concern exists
Root cause cannot be determined
OEM knowledge required
Specialized software required
Source code unavailable
Hardware obsolete
Network issue crosses IT/OT boundary
Repeated failures continue
Production risk is high
Major design modification required

Escalation is responsible engineering when the problem exceeds available information, tools, authority, or expertise.

61. WHEN TO CALL THE OEM / VENDOR

Before contacting outside support gather:

Equipment model
Serial number
PLC/HMI model
Software version
Firmware
Fault code
Symptoms
Troubleshooting performed
Relevant screenshots
Program revision
Recent changes

This reduces wasted support time.

62. TROUBLESHOOTING DOCUMENTATION

At resolution record:

SYMPTOM

What happened?

EXPECTED CONDITION

What should have happened?

FINDINGS

What was discovered?

ROOT CAUSE

Why did it happen?

CORRECTIVE ACTION

What was done?

VERIFICATION

How was successful operation confirmed?

FOLLOW-UP

What remains?

63. KNOWLEDGE ARTICLE TRIGGER

After solving a difficult issue ask:

WILL ANOTHER ENGINEER BENEFIT FROM THIS SOLUTION?

If YES:

Create/update a Knowledge Article.

Examples:

PanelView communication recovery
VFD fault recovery
PLC fault recovery
Network troubleshooting
Specific machine sequence issue
Sensor configuration
Firmware incompatibility

64. SUPPORT TICKET INTEGRATION

Troubleshooting should be linked to:

CE-SOP-002 – Support Ticket Intake, Triage & Prioritization

Record:

Ticket
Equipment
Priority
Downtime
Root Cause
Corrective Action
Resolution Time

This allows troubleshooting data to become reliability data.

65. PROJECT CONVERSION

If troubleshooting identifies required engineering improvement:

Convert/link the ticket to:

CE-YYYY-####

Examples:

New jam detection
New diagnostics
New alarm
Hardware replacement
Network redesign
Obsolescence upgrade
Sequence correction

Then follow:

CE-SOP-001

66. WEB SYSTEM — TROUBLESHOOTING TAB

Recommended sections:

INCIDENT

Ticket
Equipment
Date
Priority
Downtime

SYMPTOM

Expected
Actual
Frequency
Operating Mode

HISTORY

First Occurrence
Recent Changes
Previous Tickets

SIGNAL TRACE

Field Device
Physical I/O
PLC Tag
Mapping
Permissives
Interlocks
Command
Output
Feedback

FINDINGS

Evidence
Fault Code
Screenshots
Root Cause

CORRECTION

Temporary Action
Permanent Action

VERIFICATION

Tests
Production Validation

FOLLOW-UP

Repeat Failure
Project Required
KA Required
Upgrade Opportunity

67. WEB SYSTEM — SIGNAL TRACE TOOL

A useful troubleshooting interface would display:

INPUT SIDE

Physical Device
↓
Terminal
↓
PLC Address
↓
Mapped Tag
↓
Logic

and:

OUTPUT SIDE

Command
↓
Interlocks
↓
Output Tag
↓
Physical Address
↓
Terminal
↓
Field Device
↓
Feedback

This can turn your I/O Mapping database into a troubleshooting tool rather than merely a spreadsheet.

68. WEB SYSTEM — REPEAT FAILURE AUTOMATION

Recommended:

IF SAME ASSET + SAME ROOT CAUSE ≥ THRESHOLD

Create:

RECURRING FAILURE ALERT

Display:

Number of events
Total downtime
Previous corrective actions
Average resolution time

Then ask:

Engineering Project Required? YES / NO

69. TROUBLESHOOTING KPIs

Track:

SUPPORT EVENTS

CONTROLS DOWNTIME

MEAN TIME TO REPAIR

REPEAT FAILURE RATE

ROOT CAUSE DISTRIBUTION

BAD ACTOR EQUIPMENT

TEMPORARY VS. PERMANENT REPAIRS

TICKETS CONVERTED TO PROJECTS

KNOWLEDGE ARTICLES CREATED

70. MEAN TIME TO REPAIR

Track:

MTTR = TOTAL REPAIR TIME ÷ NUMBER OF REPAIRS

Use carefully.

The objective should not be:

Close tickets as fast as possible.

The objective should be:

Restore operation efficiently while preventing recurrence.

71. FIRST-TIME FIX RATE

Track where useful:

FIRST-TIME FIX

Issue corrected without repeat occurrence within defined period.

A high first-time fix rate may indicate strong diagnosis and corrective action.

72. TROUBLESHOOTING PRIORITY

Use:

SAFETY

↓

PRODUCTION RESTORATION

↓

ROOT CAUSE

↓

PERMANENT CORRECTIVE ACTION

↓

PREVENTION

↓

DOCUMENTATION

Emergency restoration may temporarily occur before full root-cause analysis.

Root-cause work should not simply disappear after production resumes.

73. FIVE-MINUTE DIAGNOSTIC CHECK

Before opening the PLC program ask:

1. WHAT IS THE EXACT SYMPTOM?

2. WHAT MODE IS THE MACHINE IN?

3. WHAT ALARM IS ACTIVE?

4. WHAT PHYSICAL CONDITION IS MISSING?

5. WHAT CHANGED RECENTLY?

This may solve many problems before code analysis begins.

74. PLC DIAGNOSTIC CHECK

When PLC investigation is required ask:

1. IS THE INPUT CORRECT?

2. IS THE MAPPED TAG CORRECT?

3. IS THE MODE CORRECT?

4. ARE PERMISSIVES TRUE?

5. IS AN INTERLOCK ACTIVE?

6. WHAT SEQUENCE STEP IS ACTIVE?

7. IS THE COMMAND TRUE?

8. IS THE PHYSICAL OUTPUT TRUE?

9. DOES THE FIELD DEVICE RESPOND?

10. IS FEEDBACK RETURNING?

This creates a repeatable diagnostic process.

75. THE PLC-BLAME TRAP

Avoid assuming:

"The PLC program is wrong."

because the machine stopped.

In many cases the PLC is correctly reporting or responding to:

Failed sensor
Missing air
Tripped drive
Mechanical obstruction
Missing feedback
Communication loss

Verify before modifying.

76. THE PART-SWAPPING TRAP

Avoid randomly replacing:

Sensors
Relays
Modules
Drives
Cables

without evidence.

Replacement may temporarily alter the symptom without identifying root cause.

Use measurements and signal tracing.

77. THE BYPASS TRAP

Avoid:

"Let's bypass it and see if it runs."

without understanding what protection is being removed.

A bypass may hide the symptom while increasing risk.

78. THE RESET TRAP

Avoid repeatedly pressing Reset until the machine starts.

The original fault condition may contain valuable diagnostic information.

Observe first.

Reset second.

79. THE ONLINE-EDIT TRAP

Avoid making speculative online edits during troubleshooting.

A temporary symptom may disappear while introducing an undocumented permanent risk.

If logic needs modification:

CONTROL THE CHANGE.

80. THE ONE-PERSON KNOWLEDGE TRAP

If troubleshooting depends on:

"Nathan knows how that machine acts."

capture that knowledge.

Create:

Sequence documentation
Diagnostic screens
Knowledge Articles
I/O maps
Troubleshooting notes

The department should become less dependent on individual memory over time.

81. ROOT CAUSE QUALITY TEST

Ask:

If I perform this corrective action, have I addressed why the failure happened—or only made the symptom disappear?

If only the symptom disappeared:

Continue analysis where practical.

82. TROUBLESHOOTING QUALITY TEST

Another qualified Controls Engineer should be able to review the ticket and understand:

What failed
What was tested
What evidence was found
What caused the failure
What corrected it
How the correction was verified

If not, documentation is incomplete.

83. QUICK-REFERENCE TROUBLESHOOTING WORKFLOW

1. MAKE SAFE

Protect people and equipment.

2. DEFINE

Describe exact symptom.

3. ASK

What changed?

4. OBSERVE

Capture evidence.

5. CHECK MODE

Auto/Manual/Disabled.

6. CHECK ALARMS

Find active/first-out fault.

7. TRACE INPUT

Field → PLC.

8. CHECK CONDITIONS

Permissives/interlocks.

9. TRACE LOGIC

Sequence → command.

10. TRACE OUTPUT

PLC → field.

11. CHECK FEEDBACK

Did equipment respond?

12. FIND ROOT CAUSE

Why did failure occur?

13. CORRECT

Restore safely.

14. VERIFY

Prove repair.

15. MONITOR

Watch for recurrence.

16. DOCUMENT

Record evidence and resolution.

17. IMPROVE

Prevent recurrence.

84. TROUBLESHOOTING GOLDEN RULES

RULE #1

Make the system safe before troubleshooting.

RULE #2

Define the symptom before searching for the cause.

RULE #3

Ask what changed.

RULE #4

Observe before resetting.

RULE #5

Trace signals systematically.

RULE #6

Check the physical device before blaming software.

RULE #7

Check permissives and interlocks before changing logic.

RULE #8

Command does not equal physical output.

RULE #9

Physical output does not equal feedback.

RULE #10

A force is not a permanent repair.

RULE #11

A bypass is not a permanent repair.

RULE #12

Do not make speculative production logic changes.

RULE #13

Capture fault codes before clearing them.

RULE #14

Repeat failures require deeper investigation.

RULE #15

Production restored does not necessarily mean root cause resolved.

RULE #16

Difficult solutions should become reusable knowledge.

85. DEFINITION OF SUCCESS

CE-SOP-008 is successful when Controls Engineering can consistently move from:

SYMPTOM

EVIDENCE

FAILURE LOCATION

ROOT CAUSE

CORRECTIVE ACTION

VERIFIED RECOVERY

without unnecessary program changes, random component replacement, undocumented bypasses, or dependence on individual memory.

The objective is not merely:

GET THE MACHINE RUNNING.

The objective is:

UNDERSTAND WHY IT STOPPED, RESTORE IT SAFELY, AND REDUCE THE CHANCE THAT THE SAME FAILURE HAPPENS AGAIN.

86. REVISION HISTORY

Rev. 1.0

Initial release of CE-SOP-008 – Controls Troubleshooting & Root Cause Analysis.

Owner: Electrical Controls Engineering

Review Frequency: Annual or following significant changes to troubleshooting and reliability practices.