Skip to content

Added code for CSCwo96334:/tmp directory disk space for snapshot storage#368

Open
Harinadh-Saladi wants to merge 5 commits intodatacenter:v4.1.0-devfrom
Harinadh-Saladi:tmp_dir_snapshot
Open

Added code for CSCwo96334:/tmp directory disk space for snapshot storage#368
Harinadh-Saladi wants to merge 5 commits intodatacenter:v4.1.0-devfrom
Harinadh-Saladi:tmp_dir_snapshot

Conversation

@Harinadh-Saladi
Copy link
Copy Markdown

@Harinadh-Saladi Harinadh-Saladi commented Mar 18, 2026 •
edited
Loading

Added tmp_dir_snapshot_storage_check to validate /tmp space before upgrades < 6.1(4a)
Check monitors F1527/F1528/F1529 faults on /tmp mount point only
Returns FAIL_UF if /tmp disk space issues found, NA for versions >= 6.1(4a)
Added comprehensive test cases covering affected and fixed versions
Added test fixtures for pass,fail,NA and mixed fault scenarios.

Have tested in the lab and enclosed comprehensive logs along with complete script logs.

CSCwo96334_comprehensive_Logs.txt

CSCwo96334_Pytest_Logs.txt

Complete_Script_logs.txt

dhaselva
dhaselva reviewed Mar 18, 2026
View reviewed changes
Comment thread aci-preupgrade-validation-script.py Outdated
dhaselva
dhaselva reviewed Mar 18, 2026
View reviewed changes
Comment thread aci-preupgrade-validation-script.py Outdated
lovkeshsharma702
lovkeshsharma702 reviewed Mar 25, 2026
View reviewed changes
Comment thread aci-preupgrade-validation-script.py Outdated
lovkeshsharma702
lovkeshsharma702 reviewed Mar 25, 2026
View reviewed changes
Comment thread aci-preupgrade-validation-script.py
lovkeshsharma702
lovkeshsharma702 reviewed Mar 25, 2026
View reviewed changes
Copy link
Copy Markdown

@lovkeshsharma702 lovkeshsharma702 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review comment and resubmit again please.

@Harinadh-Saladi
Copy link
Copy Markdown
Author

Harinadh-Saladi commented Mar 25, 2026

Below are the mixed fail scenario logs with /techsupport, /data/log, /firmware and /tmp directories full.

fab3-apic1# python3 aci-preupgrade-validation-script.py -d apic_disk_space_faults_check -t "5.2(8f)"
==== 2026-03-25T11-35-25+0000, Script Version v4.0.1 ====

!!!! Check https://github.com/datacenter/ACI-Pre-Upgrade-Validation-Script for Latest Release !!!!

To use a non-default Login Domain, enter apic#DOMAIN\USERNAME
Enter username for APIC login : admin
Enter password for corresponding User :

Gathering Node Information...

Current APIC Version...6.0(2j)
Lowest Switch Version...4.2(3j)

Target APIC version is overridden to 5.2(8f)

Collecting VPC Node IDs...

Progress: |████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 checks completed

=== Check Result (failed only) ===

[Check 1/1] APIC Disk Space Usage (F1527, F1528, F1529 equipment-full)... FAIL - UPGRADE FAILURE!!
Fault Pod Node Mount Point Current Usage % Recommended Action

F1529 1 1 /firmware 97% Remove unneeded images
F1529 1 1 /tmp 100% Remove unneeded logs in /tmp directory

Reference Document: https://datacenter.github.io/ACI-Pre-Upgrade-Validation-Script/validations/#apic-disk-space-usage

=== Summary Result ===

PASS : 0
FAIL - OUTAGE WARNING!! : 0
FAIL - UPGRADE FAILURE!! : 1
MANUAL CHECK REQUIRED : 0
POST UPGRADE CHECK REQUIRED : 0
N/A : 0
ERROR !! : 0
TOTAL : 1

Pre-Upgrade Check Complete.
Next Steps: Address all checks flagged as FAIL, ERROR or MANUAL CHECK REQUIRED

Result output and debug info saved to below bundle for later reference.
Attach this bundle to Cisco TAC SRs opened to address the flagged checks.

  Result Bundle: /data/techsupport/hari_test/preupgrade_validator_2026-03-25T11-35-25+0000.tgz

==== Script Version v4.0.1 FIN ====
fab3-apic1#

fab3-apic1# python3 aci-preupgrade-validation-script.py -d apic_disk_space_faults_check -t "5.2(8f)"
==== 2026-03-25T11-58-22+0000, Script Version v4.0.1 ====

!!!! Check https://github.com/datacenter/ACI-Pre-Upgrade-Validation-Script for Latest Release !!!!

To use a non-default Login Domain, enter apic#DOMAIN\USERNAME
Enter username for APIC login : admin
Enter password for corresponding User :

Gathering Node Information...

Current APIC Version...6.0(2j)
Lowest Switch Version...4.2(3j)

Target APIC version is overridden to 5.2(8f)

Collecting VPC Node IDs...

Progress: |████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 checks completed

=== Check Result (failed only) ===

[Check 1/1] APIC Disk Space Usage (F1527, F1528, F1529 equipment-full)... FAIL - UPGRADE FAILURE!!
Fault Pod Node Mount Point Current Usage % Recommended Action

F1528 1 1 /data/log 88% Remove unneeded logs in var/log/dme/log
F1529 1 1 /firmware 97% Remove unneeded images
F1529 1 1 /tmp 100% Remove unneeded logs in /tmp directory

Reference Document: https://datacenter.github.io/ACI-Pre-Upgrade-Validation-Script/validations/#apic-disk-space-usage

=== Summary Result ===

PASS : 0
FAIL - OUTAGE WARNING!! : 0
FAIL - UPGRADE FAILURE!! : 1
MANUAL CHECK REQUIRED : 0
POST UPGRADE CHECK REQUIRED : 0
N/A : 0
ERROR !! : 0
TOTAL : 1

Pre-Upgrade Check Complete.
Next Steps: Address all checks flagged as FAIL, ERROR or MANUAL CHECK REQUIRED

Result output and debug info saved to below bundle for later reference.
Attach this bundle to Cisco TAC SRs opened to address the flagged checks.

  Result Bundle: /data/techsupport/hari_test/preupgrade_validator_2026-03-25T11-58-22+0000.tgz

==== Script Version v4.0.1 FIN ====
fab3-apic1#

fab3-apic1# python3 aci-preupgrade-validation-script.py -d apic_disk_space_faults_check -t "5.2(8f)"
==== 2026-03-25T12-21-25+0000, Script Version v4.0.1 ====

!!!! Check https://github.com/datacenter/ACI-Pre-Upgrade-Validation-Script for Latest Release !!!!

To use a non-default Login Domain, enter apic#DOMAIN\USERNAME
Enter username for APIC login : admin
Enter password for corresponding User :

Gathering Node Information...

Current APIC Version...6.0(2j)
Lowest Switch Version...4.2(3j)

Target APIC version is overridden to 5.2(8f)

Collecting VPC Node IDs...

Progress: |████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 checks completed

=== Check Result (failed only) ===

[Check 1/1] APIC Disk Space Usage (F1527, F1528, F1529 equipment-full)... FAIL - UPGRADE FAILURE!!
Fault Pod Node Mount Point Current Usage % Recommended Action

F1527 1 1 /techsupport 84% Remove unneeded techsupports/cores
F1528 1 1 /data/log 85% Remove unneeded logs in var/log/dme/log
F1529 1 1 /firmware 97% Remove unneeded images
F1529 1 1 /tmp 100% Remove unneeded logs in /tmp directory

Reference Document: https://datacenter.github.io/ACI-Pre-Upgrade-Validation-Script/validations/#apic-disk-space-usage

=== Summary Result ===

PASS : 0
FAIL - OUTAGE WARNING!! : 0
FAIL - UPGRADE FAILURE!! : 1
MANUAL CHECK REQUIRED : 0
POST UPGRADE CHECK REQUIRED : 0
N/A : 0
ERROR !! : 0
TOTAL : 1

Pre-Upgrade Check Complete.
Next Steps: Address all checks flagged as FAIL, ERROR or MANUAL CHECK REQUIRED

Result output and debug info saved to below bundle for later reference.
Attach this bundle to Cisco TAC SRs opened to address the flagged checks.

  Result Bundle: /data/techsupport/hari_test/preupgrade_validator_2026-03-25T12-21-25+0000.tgz

==== Script Version v4.0.1 FIN ====
fab3-apic1#

diwakarmishra1190
diwakarmishra1190 approved these changes Apr 7, 2026
View reviewed changes
Copy link
Copy Markdown

@diwakarmishra1190 diwakarmishra1190 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally.

Note:
In the latest releases, the /tmp directory size has been reduced and snapshot dir change to /data so we will focus on checking /tmp only in the affected release.

Example:
6.0.9
bdsol-aci19-apic1# df -h
tmpfs 16G 44K 16G 1% /tmp

6.1.5
APIC-1# df -h
tmpfs 100M 4.0K 100M 1% /tmp

@Harinadh-Saladi Harinadh-Saladi changed the base branch from master to v4.1.0-dev April 9, 2026 12:24
monrog2
monrog2 requested changes May 8, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@monrog2 monrog2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lovkeshsharma702 run this through the gitlab integration test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@monrog2 monrog2 monrog2 requested changes

+3 more reviewers

@dhaselva dhaselva dhaselva left review comments

@lovkeshsharma702 lovkeshsharma702 lovkeshsharma702 left review comments

@diwakarmishra1190 diwakarmishra1190 diwakarmishra1190 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Harinadh-Saladi @monrog2 @diwakarmishra1190 @dhaselva @lovkeshsharma702