A Step by Step Continuous Integration Configuration

A step by step continuous integration configuration using SVN, Eclipse, Jenkins, Maven & Junit

1. Subversion - SVN:

Download SVN binary package from https://subversion.apache.org/packages.html suitable to your SCM (software configuration management) server platform and install SVN.

2. Configure SVN:

Configure the SVN with repository setup, user creation & SVN workflow definition.

3. Configure SVN in Eclipse:

Configure the SVN in developer's machine through their IDE - Eclipse. SVN is configured into eclipse by installing SVN plugin and SVN connector and configuring the SVN repository.

4. Install Jenkins:

Download a configurable Jenkins download from https://jenkins.io matching your CI server platform and install Jenkins.

5. User Control in Jenkins:

Create user control in jenkins using option -> manage jenkins -> configure global security -> enable security.

6. Source Control Management in Jenkins:

Configure the source control ->Configure the source control -> SVN in jenkins using option manage jenkins -> source control management section -> Please put the SVN repository url and add credential in the relevant field to access the SVN repository -> There are two options to pull committed changes from SVN.

a) Polling SCM - Polling SCM needs to be configured in SVN settings of the Jenkins.

b) Triggering build - We need to place relevant scripts for post-commit actions in hooks directory of the SVN to trigger a build remotely into Jenkins.

7. Configure Maven in Jenkins:

Configure the build tool maven in Jenkins using option manage Jenkins –> configure system –> configure maven.

8. Automated unit testing in Jenkins:

Configure Jenkins to run Junit unit cases and publish the unit test result using option configure job –> add build set up –> add post build action of executing test scripts specifying the location of test report xmls.

9. E-mail notification in Jenkins:

Configure e-mail notification in Jenkins using option manage Jenkins –> configure system –> e-mail notification section to notify post build and post unit test results to stakeholders.

10. Automated deployment in Jenkins:

Configure automated deployment in jenkins by Installing “deploy to container” plugin using option manage Jenkins –> manage plugings, then use option configure build –> “post build actions” –> “deploy war/ear to container” -> Mention the required details of target deployment server in deploy war/ear container field of the screen.

An understanding of Scaled Agile Framework

SaFe has been designed by Dean Leffingwell and his team. The purpose of SaFe is to scale the agile value from a high performing Scrum, XP and Kanban practicing team to a system and an enterprise level in an effective synchronized way.

Portfolio:

Enterprise: Enterprise represents business having portfolios driven by strategic themes as business driver.

Portfolio Management: Portfolio management owns the responsibility of strategy, program management and governance of a specific portfolio driven by decentralized decision making, continuous value flow, epic and rolling wave planning.

Epic owner: Epic owner is responsible for a portfolio level epic starting from its business case stage to approval to value stream implementation. Epic owner collaborates with multiple stakeholders.

Enterprise Architect: Enterprise architect collaborates with business stakeholders and solution architects to drive a holistic technology implementation across the portfolio value stream.

Kanban & Backlog: Portfolio management kanban addresses the flow of epic that affects value streams & agile release trains. Portfolio backlog is the highest level backlog in a portfolio.

Epic & Enablers: Epic holds the economic value of enterprise. Enablers are the technical initiative supporting business value creation at portfolio level.

Budget & Value Streams: Budget is the approved operational cost in terms of CAPEX & OPEX. Value streams are long lived systems delivering continuous value flow to customer of the enterprise.

Program:

PI Planning: Program increment planning is a 2 days release planning meeting at program level attended by agile teams, product management, architects, system team and other stakeholder. PI planning delivers a program level backlog.

ART & RTE: Agile release train is the team of agile teams that plans, commits and executes together. Release train engineer is the scrum master of the agile release train.

Business owner & Product Management: Business owners are responsible for ROI of the value delivered by a release train. Product management creates program vision and it is responsible for managing a program level backlog.

Architecture Runway: Architecture runway is a planned architecture initiative that enhances solution design, performance, cross team design and implementation. It supports emergent design of the agile team.

DevOps, Release Management, Shared Services, System Team, User Experience: These are the essential support systems enabling agile release train to deliver continuous flow of values.

Backlog, PI Objective & System Demo: Program backlog is upcoming feature and enablers. PI objectives are summarized goal that an agile release train intends to achieve. System demo is the integrated & aggregated view of the new features that an agile release train has built.

Feature & Enablers: Feature is the required system functional definition which delivers benefit. Enablers are technical initiative to support the development business initiatives.

Release any time: Release multiple times during an ART flow depending on development & business context.

Iteration:

Agile Team, Product owner, Scrum Master: Product owner is the owner of sprint backlog collaborates with program product manager and agile team. Scrum master facilitates the agile team and coordinates with other team of the agile train. Agile team is a self – managed team which turns the committed backlog into shippable value for the customer or works on a technical initiative.

XP Scrum & Kanban: XP scrum is a 2-3 week scrum iteration. Team kanban facilitates the agile team in delivering a flow of values through a workflow visualization.

Backlog & PI Objectives: Backlog is the team level prioritized backlog. PI objective is a team level business and technical goal summary.

Iteration planning, execution, demo and retrospective: Iteration planning is the sprint planning where team commits the backlog for an iteration and decomposes the committed backlog into sequenced tasks. Iteration execution is the execution of the committed task using daily scrum and Kanban board. Demo is the team level demo and retrospective is the team retrospective on the processes used in iteration.

Built-in quality: Built -in quality is an ongoing commitment & training on innovative lean-agile methods.

DevOps – A Snapshot Overview

• At Agile 2008 conference, Andrew Clay Shafer and Patrick Debois discussed "Agile Infrastructure” in August 2008.

• At O’Reilly velociry conference 2009 , John Allspaw and Paul Hammond gave their now-famous talk entitled, “10+ Deploys a Day: Dev and Ops Cooperation at Flickr.” in June 2009.

• The term "DevOps" was popularized through a series of devopsdays starting in October 2009 in Belgium.

DevOps is a culture and practice that emphasizes the collaboration and communication of software development team, QA team and IT operations team while automating the process of software delivery and infrastructure changes. It aims at creating an environment where building, testing, and releasing software can happen rapidly, frequently, and more reliably.

Collaboration, automation, continuous integration, continuous testing, Continuous Delivery, continuous monitoring, and rapid remediation are common in all successful DevOps practices.

Common DevOps Steps & Activities

• Continuous Integration -

Changes are done in small increments and changes are immediately integrated to the larger code base as soon as they are done by the developers in CI practice. Each member of the development team integrates his/her finished code at least daily into the main branch of the code. Once integrated an auto build and auto test is run. Any issue reported in auto build and auto test is addressed by the developer immediately. Notifying the integration issue and fixing the issue subsequently is kept transparent in the team. The team communicates each other more frequently. Rapid feedback, resolving the integration issue early, frequent and open communication in the team enables the team to learn faster and deliver faster.

• Continuous Testing -

Continuous testing is not only a QA function. It starts early with developers delivering an error-free code. Development also helps QA team with test data preparation and configuring the test environment as similar to production environment as possible. Operation team being aware of production usages and loading of the application helps QA to build load and performance testing. Test environments are created and configured using virtualization eliminating the cost & schedule bottlenecks. Automated testing scripts are used as much as possible to speed up the continuous testing. Small changes are tested and deployed into live constantly and development and testing processes are refined through the continuous user feedback.

• Continuous delivery -

Continuous delivery is the next logical step of continuous integration and continuous testing where small incremental changes are pushed into production constantly. Rapid and constant feedback of end-user is used for the course correction. Real-time monitoring and rapid remediation are used to minimize the failure impact. Flagging technique is used to undo and redo the changes and introducing a large change into live environment on incremental basis.

• Continuous Monitoring -

The purpose of continuous monitoring is to detect the service unavailability quickly, understand the underlying reason, apply the learning to anticipate the problems before they occur and be ready with remedial actions. Continuous monitoring start at development by using the same tools to monitor development & QA environments what are used in production environment.

Automation should be used as much as possible in testing and deployment process using DevOps tool-chain. A successful DevOps requires business, development, testing and operation teams to play significant role in each phase of the application lifecycle. A healthy collaboration among these teams is essential in eliminating the unnecessary silos.

Scaling in DevOps adoption:

Success of DevOps adoption is dependent on transforming the culture of collaboration across the teams and quickly learning from rapid feedback and experimentation. Some common steps are -

1. Improvement in quality & speed of feedback to development & testing

2. Blameless postmortems of the issues

3. Reduce the time & resources in creating functionalities from start to complete

4. Improve the repeatability in build & test processes

5. Testing should be not only automated but analyzed also

6. Remove the duplication of work in development

7. Quality and speed in test data & test environment readiness

8. Seamless flow of features from development to live environment

Continous Integration - A Snapshot

Continuous integration is an agile software development practice which facilitates in integrating the software changes frequently, stabilizes the product though auto building & self-testing builds, and uses rapid feedback to resolve the integration issues faster. The goal of continuous integration is to produce a working build as fast as possible.

A software change has been costly and risky exercise in traditional waterfall practice for many cases due to following reasons mainly –

1. Insufficient testing

2. Bug/issue detected in later stages of the project execution

3. Infrequent commit & difficult integration

4. Inflexible code base & poor version control

Development team integrates its work frequently, usually daily in continuous Integration – an agile practice. This leads to multiple integrations per day. Each integration is verified by auto building & self testing builds to detect any build error and issue with existing functionality of the software. This approach has reduced the integration problems of traditional waterfall model significantly and allowed a team to develop cohesive software more rapidly due to following reasons mainly –

1. Automated and continuous testing happens

2. Issue are detected early and addressed early

3. Regular release with better frequency

4. Better visibility of the project

Key practices of continuous integration are –

1. Maintain a single source repository

This practice facilitates auto building & self-testing of the build which leads to faster integration.

2. Auto building & self-testing builds

Build & automated testing tools are used to enable frequent build & subsequent testing and thus maintaining a clean code base by eliminating human errors.

3. Everyone in team commits everyday

This helps in testing the code development early and detecting the issue early.

4. Fix the broken build immediately

This is a very important practice needs to be maintained as a discipline. It helps faster release.

5. Notify build result to everyone in development team

Everyone is on the same page and team works in a synchronized manner.

6. Automated deployment

Eliminates human error and enables faster release.

CI Tools

There is a long list of CI/DevOps tool used across the CI/DevOps practice in software development. Some of the commonly used are –

Function	Tools
Source code Management	Git, Github, SVN, Perforce, TFS
Build Tools	Ant, Maven, Gradle ,MSBuild
Code Quality Analysis	Sonar, Coverity, Fortify
Continuous Integration	Jenkins, TFS, TeamCity, Bamboo, CruiseControl
Automated Testing	Selenium, Jmeter , Cucumber
Configuration Management	Puppet, Chef, Ansible
Continuous Deployment Tool	uDeploy, Liquibase
Continuous Monitoring	Splunk, Nagios, Hygieia

A good User Story

A software development requirement for a customer is a product value he/she has asked for. A development team views the same software requirement as work needed to deliver the product value to the customer.

User story in software development addresses the mis-communication between those who uses (user) the software and those who build (developers) it about what (value) the user wants.

3 C’s of User Story

Card - A physical representation of what a user wants in few simple plane statements.

Conversation – Details of story emerges when the developers talk with product owner and customer/customer proxies about who, what & why of the user story.

Confirmation - Objective of story identified after conversation in terms of resolved & agreed acceptance criteria.

INVEST in User Story

Independent

While conceptualizing an independent user story we should think of breaking the story into smaller vertical layers keeping the required development work in mind. This helps in prioritizing the user story in backlog queue. A dependent valuable user story can’t be implemented without implementing lesser valuable stories.

Example:

1. In a report screen, report download feature in excel and pdf should be created as two independent user stories – one for excel report and one for pdf report.

2. User validation and forgot password features in a login screen can be created as two independent user stories.

Negotiable

A user story is not a requirement. It leads to a requirement through a collaborative conversation between development team and product owner and, customer or customer proxies. Negotiation evolves the customer’s need.

Example:

1. “ I want to search an item” of user story will lead to - “I want to search an item by typing first 3 character of the item name” or - “I want to search an item by typing any 3 character of the item name” after a collaborative conversation.

2. “I want to print the letter” of user story will lead to - “I want to print the letter in a blank A4 paper” or - “I want to print the letter in a pre-printed A4 size letterhead” after a collaborative conversation.

Valuable

A user story should be of value to the user, otherwise, these is no reason for the user story to have a place in a backlog and development team to work on the user story. Value also includes something called “non-functional requirements”.

Example:

1. “As a admin user I want to create other users so that I’ll not be dependent on application support team” has a value for the user of creating login ID's independently.

2. “I want to pay by credit card” part of user story has a value of transaction to be secured as per the required regulation.

Estimable

We can’t size the user story without estimating it. If we can’t size it, development team can’t commit the story into a time-boxed sprint. Sometimes, we face an issue in estimating a user story due to its big size. The user story should be broken into smaller stories. Lack of domain & technical knowledge are the other main reasons which hinder a user story estimation. We should research in the respective areas till we are able estimate the user story.

Example:

1. “I want to purchase a product item by paying online” should be broken down into “I want to purchase a product item by paying online using net banking”, “I want to purchase a product item by paying online using credit card” and “I want to purchase a product item by paying online using COD(cash on delivery)”

2. “I want my website to be compatible with all types of mobile devices” requires a technical research spike if development team has not done this kind of the work earlier.

Small

A user story should be small enough to be delivered in one sprint depending on the size of the sprint. “Delivered” means the story gets the “done” state. Team’s capability, technology involved and methodology being used play significant role in deciding how small a story should be.

Example:

1. “I want to use touch id identification in the system” needs to be broken down into multiple user stories depending on the new functionality impact.

2. “I want to use new optional rule to submit the custom clearance application digitally” needs to be broken into multiple user stories.

Testable

Each user story must be testable in order to get it done. Testable acceptance criteria are the key aspect of making a use story testable.

Example:

1. “The application should open in IOS devices quickly” should be “The application should open in IOS mobile devices with 3G/4G connectivity in 1 second with 95% of probability under a full network strength”.

2. “The batch process of the application should complete in two hour of time between 12:00 AM to 4:00 AM” is a testable user story.

Release Planning - Step by Step

Release planning creates a plan to deliver a product value increment. This plan is called release plan and delivery team commits to the plan. Product owner, scrum master and other stakeholders support the team to achieve plan objective in their respective roles.

Who should attend Release planning meeting?

Release planning meeting should be planned and managed by portfolio management team or the designated scrum master with following invitees.

1. Scrum master – Facilitates the meeting.

2. Business Sponsor – Shares the importance of release.

3. Product owner – Represents business and owner of the product backlog.

4. Delivery team – Shares input on technical feasibility and dependencies.

5. SME & Architect – Provide on-demand expert advice

6. Stakeholders – Impacted by release who help in shaping the release plan.

Input needed

1. A ranked product backlog managed by the product owner

2. Acknowledgement if any new backlog item has been added

3. High level vision and business objective

4. Product feedback, market situation and expected deadline

5. Team’s capability, velocity and availability

6. Meeting rooms, conference bridge, hardware and software tools

7. Flip-chart, whiteboard, marker and post-it

Expected output

1. Release plan and commitment

2. Newly added product backlog item

3. Issues, risks, dependencies, and assumptions to be monitored

4. Suggestions to improve future release planning meetings

Release Planning Meeting checklist

1. Book the calendar of meeting invitees by sending the agenda in advance

2. Plan for the travel if needed

3. Book the meeting room of sufficient size with addition space for breakout sessions

4. Book video conference bridge for distributed teams

5. Arrange laptops, computer and projector

6. Arrange whiteboard, flip-charts, marker, paper, pencil and post-it

Release Planning Meeting Agenda

1. Opening the Meeting -

Scrum Mater welcomes the invitees, states the agenda & purpose of the meeting, and introduces the business sponsor. Business sponsor shares his view on importance of the release.

2. Product vision and roadmap -

Product owner share the product vision with larger picture and roadmap of the product development. Then, he/she explains how this release fits into the larger picture and roadmap reflecting the prioritized product backlog. Any new item added into the product backlog is also shared.

3. Release theme, milestone and number of iterations –

Delivery team leads the review of previous release retrospective and current status of the iterations. All participants decide a release theme & release name collaboratively. Velocity of the team if available is presented otherwise a realist velocity value is assumed. Key milestone delivery deadlines are reviewed. Timeboxes for the release are agreed. All participants come up with 1st draft to release schedule and number of iterations using the roadmap, velocity and key milestone deadlines. Team reviews any new issue & concern has come up.

4. Definition of done, Backlog Item Sizing, Story Mapping –

Team and product owner lead the review of definition of done. Definition of done is updated based change in technology and skill of the team and any other environmental factor. Next, delivery team considers items to be included for the release using the prioritized backlog. SME and product owner clarify the queries of the team around user story and its acceptance criteria which helps them in sizing the backlog item in terms of effort. Team uses different estimation techniques (planning poker is the most popular one) for backlog item sizing. Considered backlog items are mapped to iterations of the release. Larger items are split into multiple iterations. Release schedule and number of iterations are reviewed & updated if required.

5. Dependencies, assumptions and adjustments –

Technical, business, infrastructure and operation dependencies are checked. Assumptions are reviewed. Release impacted stakeholders should recheck their respective dependencies and assumption related impact. Release backlog and release schedule are adjusted accordingly if required.

6. Commit, retrospect and close -

Review of the above two steps are repeated till delivery team and product owner reaches an agreement on release backlog & release schedule. Now, it’s time to commit the release plan. Communication and logistics are planned for the release. All participants of release meeting retrospect the release planning meeting and feedback are recorded. Risk, issues, dependencies and assumptions of the release are recorded. Action plan is prepared and assigned to respective owners. Now, release planning meeting is ready to close.

Input	Steps	Output
Ranked product backlog	Plan Meeting and Logistic	Release Schedule
Vision and Roadmap	Share Vision & Roadmap	Release backlog
Business & architect inputs	Release Theme, Schedule, Iterations	Assumptions
Velocity & availability	Definition of done, story Sizing & Mapping	Dependencies
Meeting rooms	Dependencies, Assumptions, Adjustments	Risks
Tools & stationeries	Commit, Action plan, Retrospect	Feedback