[Stretch cluster] Stabilize stretch cluster tests and support CNV workloads in MON & OSD failure test cases #10580
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: mashetty330 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
pull-request-size
bot
added
size/S
pull-request-size
bot
added
size/M
There was a problem hiding this comment.
Unknown PR validation on existing cluster
Cluster Name: mashetty-st18
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
Job state: ABORTED.
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-st18
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
Unknown PR validation on existing cluster
Cluster Name: mashetty-st21
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
Job state: ABORTED.
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-st21
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_zone_shutdown_and_crash.py::TestZoneShutdownsAndCrashes::test_zone_shutdowns
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-st21
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_zone_shutdown_and_crash.py::TestZoneShutdownsAndCrashes::test_zone_shutdowns
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-st21
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_zone_shutdown_and_crash.py::TestZoneShutdownsAndCrashes::test_zone_shutdowns
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
pull-request-size
bot
added
size/L
mashetty330
force-pushed
the
sc_cnv
branch
2 times, most recently
from
8606408 to
8722c82
Compare
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-st13
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-st13
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-st13
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-st19
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-st19
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-st19
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-st19
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
Unknown PR validation on existing cluster
Cluster Name: mashetty-st22
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
Job state: ABORTED.
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-st22
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.17
OCS VERSION: 4.17
tested against branch: master
Job UNSTABLE (some or all tests failed).
mashetty330
changed the title
There was a problem hiding this comment.
Tests are passed here: https://ocs4-jenkins-csb-odf-qe.apps.ocp-c1.prod.psi.redhat.com/job/qe-deploy-ocs-cluster/44402/
Its updated as failed because of small issue with CI merge request
PR validation on existing cluster
Cluster Name: mashetty-st27
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_zone_shutdown_and_crash.py::TestZoneShutdownsAndCrashes::test_zone_shutdowns
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
Job PASSED (installation failed, tests not executed).
| replica_count=4, | ||
| namespace=constants.STRETCH_CLUSTER_NAMESPACE, | ||
| ) | ||
| break |
There was a problem hiding this comment.
why do need a break after every elif, won't a single break at the end of elif not sufficient?
| logger.info("Setting up CNV workload and creating some data") | ||
| vm_obj = cnv_workload(volume_interface=constants.VM_VOLUME_PVC)[0] | ||
| vm_obj.run_ssh_cmd(command="dd if=/dev/zero of=/file_1.txt bs=1024 count=102400") | ||
| md5sum_before = vm_obj.run_ssh_cmd(command="md5sum /file_1.txt") |
There was a problem hiding this comment.
you can use the existing function cal_md5sum_vm
| vm_obj.wait_for_ssh_connectivity() | ||
| md5sum_after = vm_obj.run_ssh_cmd(command="md5sum /file_1.txt") | ||
| assert ( | ||
| md5sum_before == md5sum_after |
There was a problem hiding this comment.
shouldn't this be checked in the test case?
There was a problem hiding this comment.
This class scoped fixture is only developed for this particular use case. The reason is because we want to perform workload verification all the tests within Test class is executed. There wont be workload specific verification after each test is executed. And only possible way for me to do that is through a separate class fixture that will setup before all the tests are executed and workload verification is done after all the tests are completed
There was a problem hiding this comment.
As discussed, please move it to the test case
| # stop the VM | ||
| vm_obj.stop() | ||
| logger.info("Stoped the VM successfully") |
There was a problem hiding this comment.
cnv_workload fixture takes care of it, still it is needed here?
There was a problem hiding this comment.
cnv_workload doesnt takes care of this. It only creates and deletes the vms.
There was a problem hiding this comment.
As discussed, please include stopping the VMs in the fixture to have an ordered way of deleting its resources
pull-request-size
bot
added
size/XL
and removed
size/L
Signed-off-by: Mahesh Shetty <[email protected]>
Signed-off-by: Mahesh Shetty <[email protected]>
Signed-off-by: Mahesh Shetty <[email protected]>
Signed-off-by: Mahesh Shetty <[email protected]>
Signed-off-by: Mahesh Shetty <[email protected]>
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-std9
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
Job UNSTABLE (some or all tests failed).
… in between Signed-off-by: Mahesh Shetty <[email protected]>
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-std9
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-std9
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_netsplit.py
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
Job UNSTABLE (some or all tests failed).
Tests that are failed are valid. Failure is not because of the automation issue. Hence its proof that its working as expected |
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-std9
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_mon_osd_failures.py
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
Job UNSTABLE (some or all tests failed).
Signed-off-by: Mahesh Shetty <[email protected]>
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-std9
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_mon_osd_failures.py
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-std9
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/functional/disaster-recovery/sc_arbiter/test_zone_shutdown_and_crash.py
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
mashetty330
added
Verified
This PR addresses: