Principles of Transaction Processing

Philip A. Bernstein, Eric Newcomer

• Publisher: Morgan Kaufmann
• Year: 2009
• Print ISBN: 9781558606234
• eBook ISBN: 9780080948416
• Edition: 2
• Topic: Computer Science
• Subtopic: Databases
• Index: Computer Science

Principles of Transaction Processing is a comprehensive guide to developing applications, designing systems, and evaluating engineering products. The book provides detailed discussions of the internal workings of transaction processing systems, and it discusses how these systems work and how best to utilize them. It covers the architecture of Web Application Servers and transactional communication paradigms.The book is divided into 11 chapters, which cover the following: Overview of transaction processing application and system structureSoftware abstractions found in transaction processing systemsArchitecture of multitier applications and the functions of transactional middleware and database serversQueued transaction processing and its internals, with IBM's Websphere MQ and Oracle's Stream AQ as examplesBusiness process management and its mechanismsDescription of the two-phase locking function, B-tree locking and multigranularity locking used in SQL database systems and nested transaction lockingSystem recovery and its failuresTwo-phase commit protocolComparison between the tradeoffs of replicating servers versus replication resourcesTransactional middleware products and standardsFuture trends, such as cloud computing platforms, composing scalable systems using distributed computing components, the use of flash storage to replace disks and data streams from sensor devices as a source of transaction requests. The text meets the needs of systems professionals, such as IT application programmers who construct TP applications, application analysts, and product developers. The book will also be invaluable to students and novices in application programming.- Complete revision of the classic "non mathematical" transaction processing reference for systems professionals- Updated to focus on the needs of transaction processing via the Internet-- the main focus of business data processing investments, via web application servers, SOA, and important new TP standards- Retains the practical, non-mathematical, but thorough conceptual basis of the first edition

Chapter 1

Introduction

1.1 The Basics

The Problem

Abusiness transaction is an interaction in the real world, usually between an enterprise and a person or another enterprise, where something is exchanged. For example, it could involve exchanging money, products, information, or service requests. Usually some bookkeeping is required to record what happened. Often this bookkeeping is done by a computer, for better scalability, reliability, and cost. communications between the parties involved in the business transaction is often done over a computer network, such as the Internet. This is transaction processing (TP)—the processing of business transactions by computers connected by computer networks. There are many requirements on computer-based transaction processing, such as the following:

■ A business transaction requires the execution of multiple operations. For example, consider the purchase of an item from an on-line catalog. One operation records the payment and another operation records the commitment to ship the item to the customer. It is easy to imagine a simple program that would do this work. However, when scalability, reliability, and cost enter the picture, things can quickly get very complicated.

■ Transaction volume and database size adds complexity and undermines efficiency. We’ve all had the experience of being delayed because a sales person is waiting for a cash register terminal to respond or because it takes too long to download a web page. Yet companies want to serve their customers quickly and with the least cost.

■ To scale up a system for high performance, transactions must execute concurrently. Uncontrolled concurrent transactions can generate wrong answers. At a rock concert, when dozens of operations are competing to reserve the same remaining seats, it’s important that only one customer is assigned to each seat. Fairness is also an issue. For example, Amazon.com spent considerable effort to ensure that when its first thousand Xboxes went on sale, each of the 50,000 customers who were vying for an Xbox had a fair chance to get one.

■ If a transaction runs, it must run in its entirety. In a retail sale, the item should either be exchanged for money or not sold at all. When failures occur, as they inevitably do, it’s important to avoid partially completed work, such as accepting payment and not shipping the item, or vice versa. This would make the customer or the business very unhappy.

■ Each transaction should either return an acknowledgment that it executed or return a negative acknowledgment that it did not execute. Those acknowledgments are important. If no acknowledgment arrives, the user doesn’t know whether to resubmit a request to run the transaction again.

■ The system should be incrementally scalable. When a business grows, it must increase its capacity for running transactions, preferably by making an incremental purchase—not by replacing its current machine by a bigger one or, worse yet, by rebuilding the application to handle the increased workload.

■ When an electronic commerce (e-commerce) web site stops working, the retail enterprise is closed for business. Systems that run transactions are often “mission critical” to the business activity they support. They should hardly ever be down.

■ Records of transactions, once completed, must be permanent and authoritative. This is often a legal requirement, as in financial transactions. Transactions must never be lost.

■ The system must be able to operate well in a geographically distributed environment. Often, this implies that the system itself is distributed, with machines at multiple locations. Sometimes, this is due to a legal requirement that the system must operate in the country where the business is performed. Other times, distributed processing is used to meet technical requirements, such as efficiency, incremental scalability, and resistance to failures (using backup systems).

■ The system should be able to personalize each user’s on-line experience based on past usage patterns. For a retail customer, it should identify relevant discounts and advertisements and offer products customized to that user.

■ The system must be able to scale up predictably and inexpensively to handle Internet loads of millions of potential users. There is no way to control how many users log in at the same time or which transactions they may choose to access.

■ The system should be easy to manage. Otherwise, the system management staff required to operate a large-scale system can become too large and hence too costly. Complex system management also increases the chance of errors and hence downtime, which in turn causes human costs such as increased stress and unscheduled nighttime work.

In summary, transaction processing systems have to handle high volumes efficiently, avoid errors due to concurrent operation, avoid producing partial results, grow incrementally, avoid downtime, never lose results, offer geographical distribution, be customizable, scale up gracefully, and be easy to manage. It’s a tall order. This book describes how it’s done. It explains the underlying principles of automating business transactions, both for traditional businesses and over the Internet; explores the complexities of fundamental technologies, such as logging and locking; and surveys today’s commercial transactional middleware products that provide features necessary for building TP applications.

What Is a Transaction?

An on-line transaction is the execution of a program that performs an administrative function by accessing a shared database, usually on behalf of an on-line user. Like many system definitions, this one is impressionistic and not meant to be exact in all its details. One detail is important: A transaction is always the execution of a program. The program contains the steps involved in the business transaction—for example, recording the sale of a book and reserving the item from inventory.

We’ll use the words transaction program to mean the program whose execution is the transaction. Sometimes the word “transaction” is used to describe the message sent to a computer system to request the execution of a transaction, but we’ll use different words for that: a request message. So a transaction always means the execution of a program.

We say that a transaction performs an administrative function, although that isn’t always the case. For example, it could be a real-time function, such as making a call in a telephone switching system or controlling a machine tool in a factory process-control system. But usually there’s money involved, such as selling a ticket or transferring money from one account to another.

Most transaction programs access shared data, but not all of them do. Some perform a pure communications function, such as forwarding a message from one system to another. Some perform a system administration function, such as resetting a device. An application in which no programs access shared data is not considered true transaction processing, because such an application does not require many of the special mechanisms that a TP system offers.

There is usually an on-line user, such as a home user at a web browser or a ticket agent at a ticketing device. But some systems have no user involved, such as a system recording messages from a satellite. Some transaction programs operate off-line, or in batch mode, which means that the multiple steps involved may take longer than a user is able to wait for the program’s results to be returned—more than, say, ten seconds. For example, most of the work to sell you a product on-line happens after you’ve entered your order: a person or robot gets your order, picks it from a shelf, deletes it from inventory, prints a shipping label, packs it, and hands it off to the shipping company.

Transaction Processing Applications

A transaction processing application is a collection of transaction programs designed to do the functions necessary to automate a given business activity. The first on-line transaction processing application to receive widespread use was an airline reservation system: the SABRE system developed in the early 1960s as a joint venture between IBM and American Airlines. SABRE was one of the biggest computer system efforts undertaken by anyone at that time, and still is a very large TP system. SABRE was spun off from American Airlines and is now managed by a separate company, Sabre Holdings Corporation, which provides services to more than 200 airlines and thousands of travel agencies, and which runs the Travelocity web site. It can handle a large number of flights, allow passengers to reserve seats and order special meals months in advance, offer bonuses for frequent flyers, and schedule aircraft maintenance and other operational activities for airlines. Its peak performance has surpassed 20,000 messages per second.

Today, there are many other types of TP applications and new ones are emerging all the time. We summarize some of them in Figure 1.1. As the cost of running transactions and of managing large databases decreases, more types of administrative functions will be worth automating as TP applications, both to reduce the cost of administration and to generate revenue as a service to customers.

Figure 1.1 Transaction Processing Applications. Transaction processing covers most sectors of the economy.

In its early years, the TP application market was driven primarily by large companies needing to support administrative functions for large numbers of customers. Such systems often involve thousands of terminals, dozens of disk drives, and many large processors, and can run hundreds of thousands of transactions per day. Large TP systems are becoming even more important due to the popularity of on-line services on the Internet. However, with the downsizing of systems has come the need for small TP applications too, ones with just a few browsers connected to a small server machine, to handle orders for a small catalog business, course registrations for a school, or patient visits to a dental office. All these applications—large and small—rely on the same underlying system structure and software abstractions.

TP systems also are being offered as services to other companies. For example, Amazon.com hosts other companies’ web storefronts. Some airlines develop and operate reservation services for other airlines. Some vendors of packaged applications are now offering their application as a service that can be invoked by a third party’s application over the Internet, which in turn helps the third party offer other TP services to their customers. Given the expense, expertise, and management attention required to build and run a high-quality TP system, this trend toward out-sourcing TP applications is likely to grow.

A Transaction Program’s Main Functions

A transaction program generally does three things:

1. Gets input from a web browser or other kind of device, such as a bar-code reader or robot sensor.

2. Does the real work being requested.

3. Produces a response and, possibly, sends it back to the browser or device that provided the input.

Each invocation of the transaction program results in an independent unit of work that executes exactly once and produces permanent results. We’ll have more to say about these properties of a transaction program shortly.

Most TP applications include some code that does not execute as a transaction. This other code executes as an ordinary program, not necessarily as an independent unit of work that executes exactly once and produces permanent results. We use the term TP application in this larger sense. It includes transaction programs, programs that gather input for transactions, and maintenance functions, such as deleting obsolete inventory records, reconfiguring the runtime system, and updating validation tables used for error-checking.

1.2 TP System Architecture

A TP system is the computer system—both hardware and software—that hosts the transaction programs. The software parts of a TP system usually are structured in a special way. As you can see from Figure 1.2, the TP system has several main components. Different parts of the application execute in each of these components.

1. End-user device: An end user is someone who requests the execution of transactions, such as a customer of a bank or of an Internet retailer. An end-user device could be a physical device, such as a cash register or gasoline pump. Or it could be a web browser running on a desktop device, such as a personal computer (PC). If it is a dumb device, it simply displays data that is sent to it and sends data that the user types in. If it is a smart device, then it executes application code that is the front-end program.

2. Front-end program: A front-end program is an application code that interacts with the end-user device. Usually it sends and receives menus and forms, to offer the user a selection of transactions to run and to collect the user’s input. Often, the device is a web browser and the front-end program is an application managed by a web server that communicates with the browser via HTTP. The front-end program validates the user’s input and then sends a request message to another part of the system whose job is to actually execute the transaction.

3. Reques...

Table of contents

• Cover image
• Title page
• Table of Contents
• Copyright
• Dedication
• Preface
• Trademarks
• Chapter 1. Introduction
• Chapter 2. Transaction Processing Abstractions
• Chapter 3. Transaction Processing Application Architecture
• Chapter 4. Queued Transaction Processing
• Chapter 5. Business Process Management
• Chapter 6. Locking
• Chapter 7. System Recovery
• Chapter 8. Two-Phase Commit
• Chapter 9. Replication
• Chapter 10. Transactional Middleware Products and Standards
• Chapter 11. Future Trends
• Glossary of Acronyms
• Bibliographic Notes
• Bibliography
• Index