I. Overview
We hope to have a clear process from development to deployment that is adhered to by every project.
We aim to reduce the reliance on developers during the testing and deployment of services, and to minimize the need for developers to make ad-hoc adjustments. The packaging and deployment process should be clear and repeatable.
We hope to have clear code branch management that facilitates version control and the ability to deploy specific versions at any time.
To address the above issues, personnel in various positions will generally need to participate in the following steps as outlined in this manual:
Ultimately, during deployment, it is hoped that it will be possible to deploy simply by using the version number of the release in question.
II. Branch Management
Popular branch management methods include GitFlow, GitLabFlow, GitHubFlow, etc., and each company may make adjustments to these processes to better suit their actual situation.
This manual adopts a different branch management process, which is based on the AoneFlow branch management process from Alibaba, which they use on their Yunxiao platform. A significant advantage of this process is that there is only one long-term existing branch, the master, and only feature and hotfix branches can commit code; the rest can only be done through merge requests, making it easy to keep the code up-to-date and stable on the main trunk.
The branch management process in this manual is adjusted based on AoneFlow, and with the foundation of previous branch management, it should be relatively easy to understand the purpose of each branch pulled out.
Please pay attention to the document description content of each branch.
In the above process diagram, in addition to the feature and hotfix branches, which can push code, the other branches can only use merge requests.
Yunxiao code management
Explanation of processes such as gitFlow
III. Development and Testing Process
(A) General Description
Based on the process outlined in branch management, the development-testing process has been organized. The following points are important in this process:
Outline Design: After receiving the task, developers first understand the desired effect of the task based on the task description, determine a feasible overall plan to achieve this final effect, and review the feasibility of this overall plan. The review of the outline design can be relatively brief as it only determines the feasibility of the plan.
Detailed Design: After the outline design has passed the review, it is refined to include table design, interface design, complex business logic description, possible impacts (historical data processing, configuration changes, etc.), and only after the detailed design review can actual coding and development proceed.
Unified Submission for Testing: Each planned minor version to be released should be uniformly ordered for testing, rather than each development team submitting for testing after completing development. If some features have significantly different development cycles and are not strongly related to each other, they can be added in batches for testing, but the same minor version, no matter how many times it is submitted for testing, should only have one submission for testing.
Unified Test Entry: When submitting for testing, all versions are uniformly ordered to the test team, which then allocates tasks internally. The purpose is to unify the testers, so that all testers have a unified understanding of the company's product functionality. Otherwise, if everyone tests independently, it may only be discovered at the end of the deployment that the functions have impacted each other.
Two email notifications in the testing process: Notification when the function is stable and can enter regression testing; Notification that regression testing has passed and deployment is approved. Both of these emails require the corresponding PM (or department manager) to perform some branch merge operations.
During the submission for testing process, each time additional content is added or a BUG is modified, as well as when testing is passed, relevant personnel need to add comments (remarks) in the submission for testing form.
In the submission for testing form, specify the version of this submission for testing, including the affected services and functions. Testing only needs to fill in the version number (currently selecting the branch) in Jenkins to deploy normally, and if it cannot be deployed, it is treated as a BUG.
(B) Plan Review
- General Requirements All development work in the product department must go through a plan review before coding can begin.
The purpose of the plan review is not to increase the workload of developers. On the contrary, after becoming familiar with and getting used to clarifying ideas before coding, considering possible coding implementation methods, and theoretically working through the coding work, the actual coding will be smoother, and there will be fewer problems encountered.
If coding work is started directly after receiving a development task, it may indeed be faster for simple functions, but for slightly more complex functions, especially those that may also depend on other services, if a detailed implementation plan is not provided in advance, and potential hidden problems and feasible solutions are not identified, then it may be difficult to solve these problems during the coding process. An even worse scenario is discovering that the previously written code may have to be re-implemented according to a new method while searching for a solution.
If a plan design is output before actual coding and reviewed, then the coding process will be clearer, knowing what to do at each step. Moreover, after the document is output, it is also better for subsequent people to understand the previously implemented functions.
Based on the above considerations, it is mandatory that all development work in the product department must pass a plan review before proceeding.
The plan is divided into two types:
Outline Design Detailed Design For development work of different complexities, the requirements for the output plan are also different, with the aim of meeting the need for a plan before coding while also reducing the complexity of the plan to a certain extent.
For simple development tasks
For example: Not involving complex business logic, just a relatively simple CRUD, this kind of development work can only output an outline design, which is a general description of what needs to be done, what interfaces need to be added or modified, and if it involves tables, it should explain the design and changes of the tables.
For more complex development tasks
For example: Involving more critical and sensitive functions (login, authorization, menu permissions, etc.), the business logic is not a simple CRUD but may have more complex logic, dependencies on other services, or the need to affect
when starting in the corresponding environment, even if no configuration is modified or added in Nacos.
Initialization data is handled using Excel. For new menus, permission configurations, and other data, after development is completed and before testing, use the Hzero export tool to export to Excel and add it to the module's data initialization directory, enabling the service to automatically perform data initialization when starting.
Table changes are managed by adding scripts to the module, which are executed automatically during deployment.
Common points for front-end and back-end:
Before submitting for testing, uniformly write a submission document, and when ordering to testing, please note the path of the submission document, submission document template.
Each time you submit for testing, you need to add comments in the test task order, correct those issues, and supplement them in the submission document.
After the regression test is passed and the go-live is permitted, move the submission document to the go-live archive folder, and modify and add some go-live content according to the template, go-live document template.
The purpose of formulating standards is to simplify deployment and not allow problems to be found during the test deployment, with developers adjusting various parameters on the fly. These issues may reappear when going live.
After the submission document is completed, order it uniformly to testing, and attach the address of the submission document.
After the order is completed, obtain the address of the test task order and fill it into the submission form.
Developers can fix issues according to their own work schedule priority after receiving the BUG order, but they must wait until the test notifies that the current round of testing is complete and can enter the next round of testing, then apply for code merging into the next round of testing.
This is to avoid the mutual impact between BUG fixes caused by too frequent deployment during the testing phase, and updating while testing will also seriously affect the overall progress of testing.
(D) Testers Once the project enters the testing phase, the direction and pace of the testing process are led by the testing personnel.
Testers can control the pace of testing to manage the developers' submission pace. This avoids redeploying new code for testing under non-blocking issues before the current round of testing is completed.
The aim is to prevent the previous issue where testers had to repeatedly deploy the environment many times a day. Previously, developers might ask testers to verify after fixing an issue.
Testers click on the submission document attached in the description to view the detailed content of this submission.
(E) Submission Process Developers prepare a unified submission document. Attach the submission document to the test task order and order it to PM.
PM creates the corresponding test branch according to the description of the submission document, merges the code, and then transfers the test task order to Wu Huangcheng.
Wu Huangcheng, based on the internal arrangement of the test team, transfers the test task to specific personnel and specifies the required working time and other information for the test.
Testers deploy the service on Jenkins and conduct testing according to the description of the submission document attached in the test task order. If a BUG is found, order it to the submission person in charge in the submission document, and the person in charge will decide who should fix this problem.
After testers complete a round of testing, if another round is needed (which may be the next round of testing or regression testing), add a note indicating which round of testing will begin and notify the developers.
After receiving the test notification to enter the next round of testing, developers supplement the submission document, then send the link of the submission document to PM to apply for a new round of branch merging, and after confirming the branch merging, add a note in the test task order indicating that the next round of testing can begin.
After testers confirm that the next round of testing can begin, start a new round of deployment and testing.
After several rounds of testing, if no new issues are found, testers add a note and notify developers that regression testing can begin.
Developers apply to PM to create a regression testing branch, and after confirming the creation of the branch, add a note in the test task order and notify testers that regression testing can begin.
Once the regression testing is passed, testers notify developers. Developers apply to PM for the closing operation (merge master, tag, delete feature), preparing for the release.
Developers transfer the submission document to the version archive path and modify the title and add relevant content for the release.
If a planned version has a large development volume or the progress is affected for some reason, batch submissions for testing can be conducted first, testing the developed functions. Just add the submission content in order in the submission form and add a note in the JIRA submission form.
IV. Release Process
Currently, our deployment has started using Jenkins. When a new service uses Jenkins for the first time, there will be some configuration work to do, but subsequent deployments can be directly deployed by selecting branches or tags.
As long as we reduce manual intervention during the deployment process and strictly follow the process, we can also avoid problems when deploying to other environments.
Regarding further automated deployment solutions, we have researched the rancher + k3s + helm chart method and plan to first set up a trial run in the development environment for a period of time, sort out and solve the problems encountered during the period. After considering it a convenient and reliable solution, then consider applying it to the production environment.