RabbitMQ in Action

Jason Williams

• Publisher: Manning Publications
• Year: 2012
• ISBN:
  9781935182979
• Topic: Ciencia de la computación
• Subtopic: Programación de código abierto

RabbitMQ in Action is a fast-paced run through building and managing scalable applications using the RabbitMQ messaging server. It starts by explaining how message queuing works, its history, and how RabbitMQ fits in. Then it shows you real-world examples you can apply to your own scalability and interoperability challenges.

Chapter 1. Pulling RabbitMQ out of the hat

This chapter covers

• The need for an open protocol—AMQP
• Brief history of RabbitMQ
• Installing RabbitMQ
• First program—Hello World

We live in a world where real-time information is constantly available, and the applications we write need easy ways to be routed to multiple receivers reliably and quickly. More important, we need ways to change who gets the information our apps create without constantly rewriting them. Too often, our application’s information becomes siloed, inaccessible by new programs that need it without rewriting (and probably breaking) the original producers. You might be saying to yourself, “Sure, but how can message queuing or RabbitMQ help me fix that?” Let’s start by asking whether the following scenario sounds familiar.

You’ve just finished implementing a great authentication module for your company’s killer web app. It’s beautiful. On every page hit, your code efficiently coordinates with the authentication server to make sure your users can only access what they should. You’re feeling smug, because every page hit on your company’s worldclass avocado distribution website activates your code. That’s about when your boss walks in and tells you the company needs a way to log every successful and failed permission attempt so that it can be data mined. After you lightly protest that that’s the job of the authentication server, your boss not so gently informs you that there’s no way to access that data. The authentication server logs it in a proprietary format; hence this is now your problem. Mulling over the situation causes a four-aspirin headache, as you realize you’re going to have to modify your authentication module and probably break every page in the process. After all, that wonderful code of yours touches every access to the site. Let’s stop for a moment though. Let’s punch the Easy button and time warp back to the beginning of the development of that great auth module. Let’s assume you leveraged message queuing heavily in its design from day one.

With RabbitMQ in place, you brilliantly leveraged message queuing to decouple your module from the authentication server. With every page request, your authentication module is designed to place an authorization request message into RabbitMQ. The authentication server then listens on a RabbitMQ queue that receives that request message. Once the request is approved, the auth server puts a reply message back into RabbitMQ where it’s routed to the queue that your module is listening on. In this world, your boss’s request doesn’t faze you. You realize you don’t need to touch your module or even write a new one. All you need to do is write a small app that connects to RabbitMQ and subscribes to the authorization requests your auth module is already publishing. No code changes. Nothing you already wrote knows anything has changed. It’s so simple a smile almost breaks out on your face. That’s the power of messaging to make your day job easier.

Message queuing is simply connecting your applications together with messages that are routed between them by a message broker like RabbitMQ. It’s like putting in a post office just for your applications. The reality is that this approach isn’t just a solution to the real-time problems of the financial industry; it’s a solution to the problems we all face as developers every day. We, the authors, don’t come from a financial services background. We had no idea what “enterprise messaging” was when we needed to scale. We were simply devs like you with an itch that needed scratching: an itch to deal with real-time volumes of information and route it to multiple consumers quickly. We needed to do it all without blocking the producers of that information ... and without them needing to know who the final consumers might be. RabbitMQ helped us to solve those common problems easily, and in a standards-based way that ensured any app of ours could talk to any other app, be it Python, PHP, or even Scala.

Over the next few chapters, we’ll take you on a ride. It starts by explaining how message queuing works, its history, and how RabbitMQ fits in. Then we’ll take you all the way through to real-world examples you can apply to your own scalability and interoperability challenges ... ending with how to make Rabbit purr like a well-oiled machine in a “downtime is not acceptable!” environment.

This is the book we wished was on the shelves when we entered the messaging wilderness. We hope it will help you benefit from our experience and battle scars and free you to make amazing applications with less pain. Before we’re done in this chapter, you’ll have a short history of messaging under your belt, and RabbitMQ up and running. Without further ado, let’s take a look at where all this messaging fun started.

1.1. Living in other people’s dungeons

The world of message queuing didn’t start out the dank and cramped one it is today, with most folks subservient to lock-in overlords. It started with a ray of light in an otherwise byzantine software landscape. It was 1983 when a 26-year-old engineer from Mumbai had a radical question: why wasn’t there a common software “bus”—a communication system that would do the heavy lifting of communicating information from one interested application to another? Coming from an education in hardware design at MIT, Vivek Ranadivé envisioned a common bus like the one on a motherboard, only this would be a software bus that applications could plug into. (See http://hbswk.hbs.edu/archive/1884.html.) Thus, in 1983 Teknekron was born. A freshly minted Harvard MBA in his hand and this powerful idea in his head, Vivek started plowing a path that would help developers everywhere.

Having the idea was one thing, but finding a killer application for it was something completely different. It was at Goldman Sachs in 1985 that Ranadivé found his first customer and the problem his software bus was born to solve: financial trading. A trader’s stall at that time was packed to the brim with different terminals for each type of information the trader needed to do his job. Teknekron saw an opportunity to replace all those terminals and their siloed applications. In their place would be Ranadivé’s software bus. What would remain would be a single workstation whose display programs could now plug into the Teknekron software bus as consumers and allow the trader to “subscribe” to the information the trader wanted to see. Publishsubscribe (PubSub) was born, as was the world’s first modern message queuing software: Teknekron’s The Information Bus (TIB).

It didn’t take long for this model of data transfer to find many more killer uses. After all, an application publishing data and an application consuming it no longer had to directly connect to each other. Heck, they didn’t even have to know each other existed. What Teknekron’s TIB allowed application developers to do was establish a set of rules for describing message content. As long as the messages were published according to those rules, any consuming application could subscribe to a copy of the messages tagged with topics it was interested in. Producers and consumers of information could now be completely decoupled and flexibly mixed on-the-fly. Either side of the PubSub model (producer/consumer) was completely interchangeable without breaking the opposite side. The only thing that needed to remain stable was the TIB software and the rules for tagging and routing the information. Since the financial trading industry is full of information with a constantly changing set of interested folks, TIB spread like wildfire in that sector. It was also noticed by telecommunications and especially news organizations, who also had information that needed timely delivery to a dynamically changing set of interested consumers. That’s why mega news outfit Reuters purchased Teknekron in 1994.

Meanwhile, this burgeoning new segment of enterprise software didn’t go unnoticed by Big Blue. After all, many of IBM’s biggest customers were in the financial services industry. Also, Teknekron’s TIB software was frequently run on IBM hardware and operating systems ... all without the boys in White Plains getting a cut. Thus, in the late ’80s IBM began research into developing their own message-queuing software, leveraging their extensive experience in information delivery from developing DB2 (see http://www-01.ibm.com/software/integration/wmq/MQ15Anniversary.html). Development began in 1990 at IBM’s Hursely Park Laboratories near Winchester, United Kingdom. What emerged three years later was the IBM MQSeries family of messagequeuing server software. In the 17 years since, MQSeries has evolved into WebSphere MQ and is today the dominant commercial message-queuing platform. During that time, Ranadivé’s TIB hardly disappeared into the bowels of Reuters. Instead it has remained the other major player in enterprise messaging, thriving through a renaming to Rendezvous and Teknekron’s re-emergence as an independent company in the form of TIBCO in 1997. The same year, Microsoft’s first crack at the messaging market emerged: Microsoft Message Queue (MSMQ).

Through all of this evolution, message queuing (MQ) software primarily remained the domain of large-budgeted organizations with a need for reliable, decoupled, realtime message delivery. Why didn’t MQ find a larger audience? How did it survive the information boom that was the late ’90s internet bubble without exper...