UrbanCode Deploy Scenarios:

What is DevOps?
Its a set of principles, practices, and products to help organizations deliver high-quality
software to market faster, while minimizing cost and risk. The IBM approach to DevOps
drives end-to-end innovation throughout the entire software lifecycle.

This still feels somewhat fuzzy to me. I am a nuts and bolts guy. I like Janitors a lot
better than Sanitation Engineers, and I like Software Testers more than I like
Quality Assurance Engineers. Lets face it, I am a Manager, not a Talent Direction
Technician. So I want a little more direct definition DevOps.

So I dug in a little deeper and looked at the DevOps Practices page. This is a little
better, it has more for me to sink my teeth into. They basically highlight five key
practices for what IBM considers DevOps. These are:

Plan, Track, and Version Everything

Dashboard Everything (my personal favorite, I am a big believer in transparency)

Automate Everything

Test Everything

Monitor and Audit Everything

That helped me sort it out in my mind. Now I think I understand the IBM definition of
DevOps. Its the blanket of improved software delivery (better, faster, cheaper) by using
the five practices listed above. That is a good starting point for me. Unfortunately, it
covers the automation, tracking, and reporting of (you guessed it) Everything. That is a
pretty massive scope. I am still wrestling with this part of the DevOps definition.

So maybe by now youve become convinced that you need to address DevOps, but
youre struggling with what DevOps is, and just what you should address first. My
advice is to talk to people in your organization, and find out what the most painful piece
of the software delivery lifecycle is. Address that piece first, and attack it in small
chunks. You need to start breaking down the technical and organizational barriers, and
share these five key practices with everyone in your organization. DevOps is a
philosophy, a way of looking at the entire software development process. By
encouraging these 5 practices in your organization, youll slowly see improvements in
how your software development is done, and begin to realize the promise that DevOps
has to offer.

Now I am lucky, because I have a dedicated team that is focused on DevOps. You can
see this in Freddys DevOps blog, in Darrells blog, and soon in Smiths blog. I can
ask them questions when I am too busy (or dumb) to figure something out. You can use
some of the same resources that I used, and you can ask our experts questions on their
blogs

The Many Layers of DevOps

For many people, DevOps and infrastructure as code are synonymous. However, I am
usually quick to point out the other moving parts that need to be managed. There is more
to DevOps than infrastructure as code. Once the infrastructure is in place we need to do
something with it. Application deployments, database updates, static content
deployments, application configuration, middleware configuration, all of these things
come into play after the infrastructure is in place.

There are many excellent applications for managing your infrastructure; tools such as
Chef and Puppet obviously come to mind (they are easily the most popular). These types
of tools are perfect for making changes across multiple servers, and making sure your
infrastructure is in sync. They are also great for taking newly spun up virtual machines
and making them identical the other servers in your environment.

Infrastructure is only part of the entire stack, and honestly it is not the most volatile part.
New machines requiring infrastructure may be added daily but once the infrastructure is
in place it tends to change less often than other components of the stack. For example,
the middleware configuration, the applications, and the application configuration all
change much more frequently than the infrastructure. Managing these changes is a
crucial aspect of DevOps.

In the image above, the lowest layers can be handled by a number of tools. These layers
are typically handled by the major IaaS providers (Amazon EC2, Rackspace, and others),
which involve provisioning of the servers, as well as the OS and the OS configuration.
Another great solution is to use Terraform or uProvision to handle the provisioning.
These tools enable companies to build an internal PaaS offering to allow teams to
automatically spin up servers and make them available for deployment.

The middleware layers have a handful of solutions for automation. Some companies
include this configuration in the base image of their virtual machines. Another popular
option is using any of the infrastructure as code tools mentioned above. These types of
configurations can even be managed as part of your application deployment in an
application deployment automation (ADA) tool such as uDeploy.

The application, and the application configuration can change multiple times a day in any
continuous delivery environment. Often it may only be the configuration itself (and not
the binaries) that need to be deployed. uDeploy is designed to not only automate the
deployment of the application but also provide visibility into which versions of changes
have been applied and where.
Getting the Most out of Your UrbanCode Deploy Implementation
Posted on November 14, 2013 by drschrag

I think it is fair to say that almost every customer that purchases or even investigates the
use of UrbanCode Deploy, or any continuous deployment solution, has already spent a
ton of time building deployment scripts, build processes, and best practices of their
own. And typically these things do the job, albeit most of the time not the most reusable
and transparent solution. So their first approach to implementing a continuous
deployment solution is to wrap their existing scripts or processes. After all, they work
and do the job, why not reuse them? Beside, there are some other features, like the
UrbanCode Deploy inventory, that are valuable in and of themselves. This is considered
the low hanging fruit implementation of something like UrbanCode Deploy. Simply
wrap what you have and you are off and running.

But if you peek just a bit into the future, this is not the best approach. The following are
some common shortcuts that people take instead of implementing with future benefits in
mind.

Simply executing shell scripts to perform deployments this is the quick and
dirty solution, but misses out on a number of valuable capabilities. First, you
should take advantage of as many existing plugin steps as possible. Plugins offer
canned and consistent execution of a single function. Reusing a step that you
know works insure that you dont have random errors across different deployment
processes. I would even argue that you should be on the lookout for abstracting
process steps into custom plugins. If you think there might be some reuse of a
step that you have scripted, find a way to wrap it in a plugin step.
Also, upgrading a plugin upgrades the use of those plugin steps everywhere in
your UrbanCode Environment. If you find a better way to execute a step, upgrade
the plugin and all deployment processes that use that step get the new and
improved way.

Not taking advantage of templates It is understandable that you would want to

build a deployment process and test it first before you would consider it the
 default way. However, you should quickly abstract processes into templates to
insure best practices are followed. If you have developed a tried and true way to
deploy apps to Tomcat, for example, then you should provide a way for each and
every Tomcat application to take advantage of your success. Creating component
templates that encapsulate standard processes is something that should be put at
the forefront of an implementation effort.

Not taking advantage of UrbanCode Deploy code station Code station is

UrbanCode Deploys internal component content storage mechanism. Many
customers already have some type of build target repository (i.e. Nexus) and dont
see the need to replicate this storage also in UrbanCode Deploy. But code
station is more than simply a repository. It also hashes each and every file that
makes up a component. This provides an easy mechanism to know if files have
changed when a new version of a component is provided. A key value of
UrbanCode Deploy is that it only updates the files that have changed, and you
dont need to do anything to enact this feature. It accomplishes this by comparing
hashes. If the file has not changed and the inventory indicates it already exists,
then there is no reason to push it again. Of course you can override this and push
it anyway, but why would you?

Providing only changed elements to UrbanCode Deploy many customers have

been bitten in the past by overwriting code that works with code that doesnt.
Almost everyone has a story that involves many hours of hunting for the cause of
production problem only to discover that something that worked got overwritten
by something that was broken and that element being overwritten wasnt even
supposed to be affected by the changes. The natural knee-jerk reaction to this is
that only things that you believe have changed get deployed. The mentality goes
way back to mainframe days. Providing complete builds to UrbanCode Deploy
allows you to take advantage of the code station feature as described above, and it
allows you to always be able to deploy any version since UrbanCode always
knows how to get from full component version to full component version.

Most implementation of a continuous deployment solution start out with a quick win and
hopefully everyone can be back on to their regular job. But by spending a small
amount of time to do things right from the beginning, there is a lot of value to be had in
the future.
Promoting a Multi-tiered App from QA to Production

In this scenario, we will look at taking a typical web application and promoting it through
several environments. The different tiers will allow us to review different types of
deployments. We will also examine how those related deployments are coordinated to
deploy the entire application. Please find detailed discussions of different parts of this
promotion and deployment process along the side of this page.
The Application
For this example, let's declare our application to be a simple customer facing hotel
booking system. Customers will come to the website, search for available inventory, and
book a stay. Business rules and inventory are maintained in a back-end database.
Meanwhile, images and dynamic content are provided by the art team and deployed to
dedicated front end web servers. The main application is deployed to an application
server. We will ignore the specifics of which database system, front end servers and
application servers are used UrbanCode Deploy provides a wide array of integrations
to handle integration with external tools.
Example Environments
While UrbanCode Deploy is capable of managing a virtually unlimited number of
environments for each application, the example application will move through just a
handful of environments. QA: A classic early test environment used mostly for automated
tests User Acceptance: A test environment dedicated to manual testing of business
requirements Stage: A production-like environment used for final testing

Production:
The Live environment customers access Disaster Recovery: A smaller clone of
production in another data-center that is used for failover when production goes down
Snapshots Our application is made of a database back-end, a middle-ware tier and web
front-end. Each of these components could be delivered by a different team, making it
easy to mix and match versions. However, changes to several components are related and
should be deployed together. This deployment dependency needs to be managed.
UrbanCode Deploy uses the idea of the "Snapshot" to tie together versions that are
known to work well together. A snapshot wraps these known compatible versions into a
desired deployment state. At deployment time, UrbanCode Deploy analyzes what is
already deployed, and makes the needed changes to the target environment.
Managing What Gets Deployed

Traceable deployments benefit from an artifact repository, which plays some key roles in
the deployment: Holding the files to be deployed. Ensuring the files are not tampered
with. Establishing rules for cleaning up old versions. Access control. UrbanCode Deploy
Has an Integrated Artifact Repository Artifact repositories are such a critical element of
deployments that UrbanCode Deploy comes with a built in artifact repository,
CodeStation.
This is beneficial since the user does not have to configure another system to store the
files to be deployed. Also, the integrated artifact repository has features such as security
and clean-up policies to efficiently manage disk space. Another feature of the artifact
repository is automated artifact hashing to protect against corruption or tampering with
files.
How Is It Populated?
There are several ways of populating the artifact repository. In our example hotel booking
application, the core business logic (the middleware code) is populated automatically by
a promotion out of its build system, AnthillPro. Database updates are pulled directly out
of source control ad hoc. The front-end content is retrieved by UrbanCode Deploy from a
drop-box used by the art team and pushed into the repository. Configuration changes to
the middleware are extracted from either the QA or Staging environments. Regardless of
how the component versions are populated, UrbanCode Deploy retrieves them from the
repository and tracks their versions as it deploys them. Integrating with Other
Repositories
Already have an artifact repository that you know and love? No problem. UrbanCode
Deploy provides integration hooks so that it can use alternate repositories.
Incremental Content Deployments
When deploying content, database changes and even some applications, incremental
deployments are common. Incremental deployments can be much more efficient in
circumstances where only a few hundred kilobytes out of a total of hundreds of
megabytes have changed. Efficiency is only one reason to deploy incrementally. Some
deployment types are almost always incremental. Database updates are a classic example.
Rarely will a schema-change result in dropping the database and recreating it. Instead, it
is normal to apply a handful of ordered updates changing the database in place. For these
reasons, UrbanCode Deploy comes with robust support for incremental deployments.
How Does UrbanCode Deploy Handle Incremental Deployments?
UrbanCode Deploy supports full and incremental packages (packages are component
versions). It knows whether a specific component version is incremental or full. The
component's deployment process smartly handles each type: For example, in our content
deployment the flat file deployment process knows that incremental packages contain
files that are added and modified, as well as an optional manifest of files that are to be
deleted. In the case of a full deployment, the current content would be replaced with the
new content.
Handling Multiple Incremental Changes
With incremental updates to the content and database portions of our booking application,
it's typical that multiple incremental updates will be made to the QA environment before
a promotion to Stage or Production. UrbanCode Deploy automatically rolls up all the
pending incremental updates into a single deployment to a later environment. In fact, it
checks each server's inventory to determine which changes are needed and pushes only
those.
Deploying to Application Servers
Application Server and Middleware Support
UrbanCode Deploy provides integrations with many of the most popular Application
Servers, including both Java and Microsoft technologies, as well as related middleware
tools like message brokers. UrbanCode Deploy supports both middleware application
component deployments, and the migration of the configuration, to these tools.
Middleware Code Deployments
In our example booking system, the middleware code is a compiled application; perhaps
represented by an EAR file for Java, or an ASPx application if it is .Net. Typically, these
types of applications are deployed using the Full Deployment methodology. A new build
is provided by the build system (like AnthillPro) to UrbanCode Deploy, and the entire
build is deployed each time. The deployment may involve stopping the web server (or the
application within it), moving files, and restarting. Many web servers provide remote
APIs that UrbanCode Deploy uses to perform this remotely. For others, UrbanCode
Deploy uses agents installed on the target machines to perform the update. Already have a
great deployment script? No problem. In additional to out-of-the-box integrations,
UrbanCode Deploy can use any script you already have. Some teams perform
incremental updates of the middleware code overwriting only the changed files within
the application. This is fully supported by UrbanCode Deploy.
Middleware Configuration Deployments
UrbanCode Deploy can take declarative configuration changes and apply those changes
to a target environment. Tailored integrations with the specific middleware tools enable
this seamless promotion. Like the code, either incremental changes (a declarative change)
or full deployments are supported. In both cases, environment-specific parameters are
taken into account and substituted on the fly. For example, in configuring a database
connection, many of the options may be the same in all environments, but the database
server name and connection credentials may differ. UrbanCode Deploy will substitute the
server name and credentials for the target environment.
Deploying to Databases
Like many applications, our hypothetical booking application has a database back-end.
Without automation of any sort, our developers would submit SQL scripts and stored-
procedure updates to the DBAs, along with instructions for how they should be run and
where. These types of hand-offs are error prone and the updates lack traceability. Built-in
Database Deployment Kit
Database deployments are a particularly tough challenge. To meet it, UrbanCode Deploy
provides an easy to use the Database Deployment Kit out of the box. This tool set allows
the database developers to explicitly define ordering dependencies between changes.
They can have one set of changes that originated in a maintenance branch and separate
collections of changes supporting new features. What Version Are You On? At
deployment time, UrbanCode Deploy interrogates the database to see which changes
have already been applied. Then (in order) applies the remaining updates.
Consistent Database Deployments across Environments
Because UrbanCode Deploy uses the same process to deploy the database across all
environments, our example application will have tested the database update in QA,
Performance, UAT and Staging prior to the Production deployment. This will give us a
high degree of certainty that the production database deployment will work perfectly.
Creating and Promoting Snapshots
Our example application is made up of a database, middleware configuration,
middleware code, and static content. A release will often deploy specific versions of
multiple components. UrbanCode Deploy ties together versions of these various
components into a "Snapshot". A snapshot allows us to deploy the same set of
components through our higher environments (like Performance, UAT, Staging, and
Production) ensuring that we continue to deploy the same components that were tested
together earlier. This prevents the problem of applications failing in production because a
component another part of the system was dependent on was left behind. UrbanCode
Deploy understands deployment dependencies.
Creating Snapshots
The simple way to create a snapshot is to base it on an environment. Essentially, the user
would be saying, "I like what is in QA ... that is what I want to promote to Performance)."
UrbanCode Deploy examines the inventory of the QA environment and constructs a
snapshot for the user to review. Snapshots can also be constructed ad hoc by manually
picking and choosing the versions that should be deployed together. Deploying Snapshots
A snapshot can be deployed to target environments by users who have permission to the
application and target environment. A snapshot deployment will always move the entire
snapshot, although components which are already of the correct version in the target
environment may be automatically skipped.
Snapshot-only Environments
UrbanCode Deploy provides a distinction between environments that support ad hoc
deployments of individual components and those that can only accept snapshot
deployments. While any environment could be of either type, often one or two early test
environments support ad hoc deployments while later environments and production are
restricted to snapshot-only deployments for safety.
Approval Gates
The promotion of a snapshot to an environment can be gated by approval. Users can
submit snapshots for approval, for example a deployment to Production. Each
environment requiring approval is associated with one or more approval roles. When an
approval is requested, members of each approval role are notified and they can approve
the snapshot deployment right inside UrbanCode Deploy. When all members have signed
off, that snapshot is freed to be deployed to the target environment by the appropriate
users.
How Does UrbanCode Deploy Handle Rollbacks?
UrbanCode Deploy comes with a Package Management System (PKS). The PKS knows
how to install different types of packages: native packages such as MSIs and RPMs; JEE
packages such as WARs and EARs; packages of database changes, or configuration
changes; as well as flat file and many other types of packages. The support for each
package type knows how to perform a rollback for that specific package type. For
example, rolling back an EAR-file deployment requires redeploying the previous EAR
file; a rollback of a flat-file deployment requires restoring all the original files that were
deleted or changed and the removal of all files that were added by the deployment being
rolled back. Rolling back database changes involves the execution of the user provided
roll-back scripts.
Proven, Built-in Rollbacks
The roll-back strategies employed by UrbanCode Deploy are standard, and a good thing.
UrbanCode Deploy uses are tried and true rollback strategies, providing their
implementation without the need for developing and maintaining any scripts that
implement rollback logic.
Using UrbanCode Deploy to deploy to Microsoft IIS
UrbanCode Deploy provides integration with a number of Microsoft technologies out of
the box. In a typical scenario, an application might have several interrelated websites,
web services, and database components. Each of these components may be built in a
Team Foundation Server or another build solution. UrbanCode Deploy will detect new
builds automatically, and load them as component versions available for deployment.
Deployments can be of individual components or of Snapshots which use a model driven
approach to deploy the entire application (or just the components which changed).

IIS Integration
The UrbanCode Deploy IIS integration, provides support for loading new website
versions, performing routine restarts with application or pool level starts and stops, and
configuring IIS itself. That configuration can include creating and deleting applications,
pools, virtual directories, and sites. Managing pesky web.config and app.config files
Many applications have configuration files of some sort that need to be modified on a per
environment basis. UrbanCode Deploy has this covered. When you define the
environments to deploy to, you can declare variables (properties) that change
environment to environment. Users who own each environment can set the correct values
for that environment. From there, UrbanCode Deploy can either edit the configuration
files at deployment time with environment and server specific information, or create the
whole file from a template