Dynamic CI Pipeline 13 December 2016.

Apresentação em tema: "Dynamic CI Pipeline 13 December 2016."— Transcrição da apresentação:

1 Dynamic CI Pipeline 13 December 2016

2 Problem Statement No flexible assignment of test to resources
Installer to POD Scenario to Installer Testcases to Scenario Verification / testcategories to scenarios Definition of such mapping cannot be done easily and quickly by the user Scenario/Installer to POD mapping is static Scenario details coded by installer Testcase to scenario mapping coded by testing Tests to categories Scenarios to projects? Typically projects create new scenarios together with testcases. Is it necessary? 12/11/2018

3 Principles / Goals Resources should be allocated dynamically
Users select a resource from a pool checking the resource capabilities CI allocates resources with different capabilities and assigns jobs to a suitable resource Users specify their requirements in their own space Projects define testcases and testcase description in their repo Testcase description contains all information about when the testcase can be run (category, scenario-prerequisites, POD-prerequisites) POD owners describe all capabilities of their resource Scenario owners describe capabilities and prerequisites of the scenario CI, Installers, Testframeworks can read these descriptions and dynamically do the mapping [POD;Installer;Scenario;Testset] 12/11/2018

4 One more thing .... A deployment could be used by multiple test runs in sequence A build creates artifacts that can be used for multiple deployments Not every execution of the pipeline needs to start from scratch 12/11/2018

5 CI Pipeline Dedicated feature test build deploy test Silver category
User defined Jobs: Dedicated feature test Verification Jobs: Pre-merge Post-merge Periodic Jobs: Daily / weekly build deploy test Silver category Gold category Platinum category Release verification Code verification Deploy verification Testcase verification artifacts scenarios 12/11/2018

6 How do we get there? (1) Common configuration files
Data modeling of POD resources and capabilities Scenario definition files Data modeling of deployable components and their configuration Requirements of scenarios on PODs (e.g. Testcase descriptions Feature that is tested Requirements on system-under-test (deployed scenario) Test category CI needs to create a valid task: Build/deploy/test steps to execute According to trigger/test category/etc. Select scenario Find one that provides all features for the tests Select installer Any installer that can deploy the scenario Select POD That has all prerequisites for tests, scenario and installer Build step creates artifacts Installer reads POD configuration and Scenario definition to execute deployment Create additional deployment infos as input for test Testframework executes selected set of tests 12/11/2018

7 How do we get there? (2) Verification and periodic jobs as defined in „CI Evolution“ Add Cross Community CI (3rd party CI) Triggers also from upstream gerrit Periodic jobs for stable and also for latest upstream versions Provide flexibility for user jobs: Select sources/artifacts Select scenarios/installers Select features/testcases 12/11/2018

8 How do we get there? (3) Stepwise approach necessary
Lab owners create POD configuration files for all PODs Installers can move to use those as soon as they are ready, but dynamic assignment to PODs is only possible if they have implemented it. In the meantime, installers can manually convert POD configuration files to their specific need when they need to switch to a differnt POD Create Scenario definition files for all scenarios before E release We can do manual conversion to installers where dynamic assignment is not possible yet Wherever a scenario definition file specifies „new things“ these still have to be implemented in installers (e.g. New OpenStack modules to be deployed) Testing team already works on test categories Test description needs to be readable by CI, so scenario/resources/installer can be selected Work on scenario consolidation in parallel Scenario definition files help to be able to merge scenarios 12/11/2018

9 Scenario Compatibility Map
Scenario Compatibility Map? (based on colorado scenario list, danube has more) os-nosdn-nofeature-ha os-nosdn-nofeature-noha os-nosdn-kvm-ha os-nosdn-kvm_ovs-ha os-nosdn-kvm-noha os-nosdn-kvm_ovs-noha os-nosdn-ovs-ha os-nosdn-ovs-noha os-nosdn-vlan-ha os-odl_l2-nofeature-ha os-odl_l2-sfc-noha os-odl_l2-sfc-ha os-odl_l2-bgpvpn-ha os-odl_l2-bgpvpn-noha os-nosdn-fdio-noha os-odl_l2-fdio-noha os-odl_l2-fdio-ha os-odl_l3-fdio-noha os-odl_l3-fdio-ha os-odl_l3-nofeature-ha os-odl_l3-nofeature-noha os-odl_l3-vpp-ha os-ocl-nofeature-ha os-ocl-nofeature-noha os-onos-nofeature-ha os-onos-sfc-ha os-nosdn-lxd-noha os-nosdn-lxd-ha Observations: Too many fields There are groups of scenarios which contain each other Do we need everything in ha+noha?

10 Ideas Some scenarios are needed for development
Simple basic environment for installer verification Start with noha, add ha later Start with nofeature, add features later Customer view wouldn‘t need some scenarios: noha scenarios, that are “contained“ by others Do we need to release “contained“ and noha scenarios? With CI evolution, can we reduce weekly builds to only the scenarios that will be released? 12/11/2018

11 Scenario Compatibility Map
Scenario Compatibility Map? (based on colorado scenario list, danube has more) os-nosdn-nofeature-ha os-nosdn-nofeature-noha os-nosdn-kvm-ha os-nosdn-kvm_ovs-ha os-nosdn-kvm-noha os-nosdn-kvm_ovs-noha os-nosdn-ovs-ha os-nosdn-ovs-noha os-nosdn-vlan-ha os-odl_l2-nofeature-ha os-odl_l2-sfc-noha os-odl_l2-sfc-ha os-odl_l2-bgpvpn-ha os-odl_l2-bgpvpn-noha os-nosdn-fdio-noha os-odl_l2-fdio-noha os-odl_l2-fdio-ha os-odl_l3-fdio-noha os-odl_l3-fdio-ha os-odl_l3-nofeature-ha os-odl_l3-nofeature-noha os-odl_l3-vpp-ha os-ocl-nofeature-ha os-ocl-nofeature-noha os-onos-nofeature-ha os-onos-sfc-ha os-nosdn-lxd-noha os-nosdn-lxd-ha Observations: Too many fields There are groups of scenarios which contain each other Do we need everything in ha+noha?

12 Reduced Compatibility Map
Reduced to 11 top-level scenarios Noha only where necessary Combine obvious features Next steps Can we release only these? Can we find compatibilities to combine further? L2 & L3 ? Bgpvpn & sfc ? Fdio & vpp os-nosdn-kvm_ovs-ha os-nosdn-vlan-ha os-odl_l2-sfc-ha os-odl_l2-bgpvpn-ha os-nosdn-fdio-noha os-odl_l2-fdio-ha os-odl_l3-fdio-ha os-odl_l3-vpp-ha os-ocl-nofeature-ha os-onos-sfc-ha os-nosdn-lxd-ha 12/11/2018

13 Next steps Do same analysis for 58 danube scenarios
Check concrete proposals with scenario owners No scenario shall be deleted, but If all features of a scenario can be tested also in another scenario, we can pause to use the first scenario until release date. Keep one simple scenario for sandbox usage 12/11/2018