By Rich Ptak
Enterprise
transformation to become a Digital Enterprise has received a lot of attention
since BMC started the discussion with its DSM initiative. Enterprises are
transforming operations to digitize functions and products in response to
customer demands for speedier, innovative and personalized services. To meet
customer expectations, apps defining services must be rapidly developed,
continuously updated and digitally deployed.
The
services themselves must be intuitive, user focused, constantly available on
any user device especially cloud and mobile devices. A financial services
consumer may want to check an account balance, analyze market trading, examine
a company’s profile and transfer funds before completing a stock trade using
one or a combination of devices. A single transaction spins up multiple
transactions accessing a number of diverse data sets. The service must be
available 24/7, operating flawlessly at rapid speeds. Anything less risks a
dissatisfied customer and loss of business. The process is repeated across
thousands, even millions of customers.
More
sophisticated analytics along with different data mean that traffic volumes,
data manipulation and analysis are expanding with no end in sight. Data center
operations are hard-pressed to keep up with this activity with existing
infrastructure management tools.
With
some 70% of enterprise data residing on mainframes, most of the activities will
involve mainframes working across multiple subsystems with shared data residing
on both the mainframe databases along with other systems. Optimal database performance
requires frequent and often time and storage intensive data base reorgs and
clean-ups. With increasing size, these become progressively more difficult,
more disruptive, and more resource intensive.
Already
stressed, data centers are being pushed to operate with minimal or even no
downtime. Unfortunately, traditional data management tools were not designed
for the current digital reality, including such frequent and persistent updates
and reorgs of massive amounts of data. These tools require data bases to be
off-line and require intensive resources that grow uncontrollably as the data
volumes grow. These issues often require
IT to delay or perform partial maintenance, which only slows performance deterioration,
but doesn’t often improve performance. Additionally, forced downtime incurs
added costs such as lost/delayed business, loss of customers turning to other
suppliers, disrupted operations, etc.
Expensive
alternatives (capacity over-provisioning, manual scripts to make temporary
repairs and adjustments, equipment upgrades) can maintain operations until a
full reorg is run to help avoid costly penalties due to service agreement
breaches. However, these “work-arounds” have their
own draw-backs and costs.
BMC’s
Next Generation Technology Utilities for DB2 (NGT) are designed for the
challenges of digital business. NGT intelligently automates and monitors the utility
process and runs with the data base on-line and fully
operational. We interviewed a senior engineer responsible for data
center architecture and operations at a financial services firm on his
experiences with NGT.
The Corporate Data Center
Our
senior engineer is responsible for operations in a mainframe shop consisting of
multiple sysplexes totaling approximately 80K MIPs processing capacity. Their
database runs 60 to 70 Terabytes after compression. Operating 24 hours/7 days per
week, smooth, continuous operations is critical to the success of the business.
The
company faced challenges impacting many DB2 environments. Their data bases were growing rapidly with
critical ones early each week reaching sizes that seriously impaired performance.
They were seeing 1000 timeouts each week during data management processing. The
workload would not allow downtime for a reorg during the week so optimization
was delayed to the weekends. Temporary adjustments and workarounds made manually
during the week were less and less effective as the workload grew. Using available
tools, the only alternative was to let performance deteriorate until weekends and
then, squeeze-in reorg and maintenance. Even then, traditional management techniques
took too much time, decreasing availability (risking expensive service
agreement violations) and ran up huge costs.
IT
staff believed automation would be too costly and difficult (if not impossible)
to successfully implement as many of the 10’s of thousands of jobs run followed
no DB2 object naming conventions. This meant complex, expensive, time-consuming
manual customization of reorgs specifying sort lists, setting parameters, handling
overrides, and tuning. They were happy to find out that with NGT, this was not true.
The Financial
Service Firm’s Experience
The
IT staff used existing management solutions from BMC and IBM for ‘on-line’ data
management. They needed a better solution.
As mentioned, they were initially skeptical an automated management approach
could work because of the complexity of their naming and workflow processes. Our
engineer decided to try BMC’s NGT solution.
NGT allows reorg processes to be run in real-time
without disrupting or affecting service delivery operations. Thus, it assures near
peak data base performance at all times.
NGT
was able to tie into the existing "grouping" system and determine which
ones need to be reorged on a daily basis. Placing jobs in groups simplifies
reorg management, e.g. objects are movable from group to group via a simple “drag ‘n drop”. Thousands of jobs were replaced
by wildcarded or "group" reorgs that automatically and dynamically
handle the whole system.
Provided basic parameters and policies, NGT
schedules and automatically runs reorgs as necessary. The utilities ‘learn’ what works and what
doesn’t. It ‘remembers’ what has been executed and monitors process
effectiveness. It uses this data to automatically make adjustments to executing
processes. This assures that reorgs are done in the most efficient, rapid and
optimal manner. Required reorg jobs went from tens of thousands to between
five and six thousand jobs. Now, there are many fewer exceptions requiring
overrides. Where a standard job ran with thousands of overrides, they now have
fewer than 100. Instead of delaying until the weekends, reorgs are now run as
often as they need or wish.
NGT
Utilities provides full data base availability, improved application
performance and fully automated data management housekeeping. Dataset
allocation automatically adjusts to fit business policies, saving time and
storage space. One unforeseen benefit, built-in integrity checks revealed database
issues involving thousands of corrupted indexes. The corruption had been masked
because of an automatic rebuild process in their existing solutions.
Prior to NGT, they were able to manage only about 20% of their system because of its size and complexity. NGT enables them to run approximately 6 times as many reorgs at reduced costs while automatically managing the entire system. By being able to reorg through out the week, they maintain good performance and overall save about 10%.
Advice
Our engineer was impressed with NGT’s ease of use. He
cautioned that unlike traditional tools, it doesn’t generate a lot of real-time
status reporting. Experienced DB staff may have difficulty adjusting to its
lack of in-process reports. Not to worry, automatically it does a lot more behind
the scenes. Time and detailed post-reorg reports do much to eliminate that
concern.
For example, upon completing a reorg, NGT provides a very
detailed activity report and error report summary. The summary provides useful insights
into what was done in the reorg which serves as a basis for further action. Finally, it is possible to insert exit points
into the utilities to customize the response to specific situations. For
example, NGT can be coded to alter a table entry in response to a fault or
error instead of generating a Trouble Ticket.
Conclusion
The
senior staff engineer found the potential for continuous, automated
reorganization invaluable. NGT is an incredibly powerful solution that is able
to complete in much less time without interrupting operations. He liked how it
learns and improves its effectiveness with each reorg. He found it highly
useful and reliable in operation.
The
volume of post-reorg data can be intimidating. But, as stated before, powerful
NGT utilities more than compensate for this. They do their job automatically
and well by providing insights into significant hidden or previously unknown
problem areas. All without ever taking the database off-line. An excellent solution
to the major challenges faced by this financial services firm.
No comments:
Post a Comment