Troubleshooting OpenStack

Tony Campbell

• Publisher: Packt Publishing
• Year: 2016
• eBook ISBN: 9781783986880
• Edition: 1
• Topic: Computer Science
• Subtopic: Virtualisation
• Index: Computer Science

Get unstuck and start stacking!

About This Book

• Easily fix the nagging problems that commonly plague OpenStack and become the go-to person in your organization
• Get better equipped to troubleshoot and solve common problems in performance, availability, and automation that confront production-ready OpenStack environments
• Save time and decrease frustration by solving significant issues that arise from OpenStack deployments pertaining to storage and networking

Who This Book Is For

You will need a basic understanding of OpenStack, Linux, and Cloud computing. If you have an understanding of Linux, this book will help you leverage that knowledge in the world of OpenStack, giving you confidence to tackle most issues that may arise.

What You Will Learn

• Diagnose and remediate authentication and authorization problems in Keystone
• Fix common issues with images served through Glance
• Master the art of troubleshooting Neutron networking
• Navigate and overcome problems with Nova
• Troubleshoot and resolve Cinder block storage issues
• Identify and correct Swift object storage problems
• Isolate and fix issues caused by Heat orchestration
• Leverage Ceilometer and other metering and monitoring tools for effective troubleshooting

In Detail

OpenStack is a collection of software projects that work together to provide a cloud fabric. OpenStack is one of the fastest growing open source projects in history that unlocks cloud computing for everyone. With OpenStack, you are able to create public or private clouds on your own hardware. The flexibility and control afforded by OpenStack puts the cloud within reach of anyone willing to learn this technology.

Starting with an introduction to OpenStack troubleshooting tools, we'll walk through each OpenStack service and how you can quickly diagnose, troubleshoot, and correct problems in your OpenStack. Understanding the various projects and how they interact is essential for anyone attempting to troubleshoot an OpenStack cloud. We will start by explaining each of the major components and the dependencies between them, and move on to show you how to identify and utilize an effective set of OpenStack troubleshooting tools and fix common Keystone problems. Next, we will expose you to common errors and problems you may encounter when using the OpenStack Block Storage service (Cinder). We will then examine Heat, the OpenStack Orchestration Service, where you will learn how to trace errors, determine their root cause, and effectively correct the issue.

Finally, you will get to know the best practices to architect your OpenStack cloud in order to achieve optimal performance, availability, and reliability.

Style and approach

This is straight-to-the point guide to fixing your OpenStack cluster. Common problems are identified and suggestions to resolve these problems are presented in a simple, easy-to-understand manner.

Troubleshooting OpenStack

---

Table of Contents

Troubleshooting OpenStack

Credits

About the Author

About the Reviewer

www.PacktPub.com

eBooks, discount offers, and more

Why subscribe?

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Errata

Piracy

Questions

1. The Troubleshooting Toolkit

The project overview of OpenStack

Keystone

Glance

Neutron

Nova

Cinder

Swift

Heat

Ceilometer

Horizon

Oslo

Documentation

Ironic

Magnum

Trove

Barbican

Congress

Designate

The supporting technologies

Linux

Databases

Message queue

The Apache web server

Basic troubleshooting methodology and tools

General Linux tools

Linux processes

ps

pgrep

pkill

top and htop

Hard drives

df

fdisk

parted

cat /proc/partitions

Installed packages

General tools

The watch command

File tools

Message broker tools

RabbitMQ

Summary

2. Troubleshooting OpenStack Identity

Know your version

Running Keystone under Eventlet

Checking the Keystone service

Checking the Keystone client

Checking the OpenStack Client

The client debug mode

Checking the API

Keystone process not starting

Database stopped

The service catalog endpoint

Running under WSGI

mod_wsgi

wsgi-keystone.conf

Stopping the Eventlet process

Checking WSGI files

Checking the Keystone service

Summary

3. Troubleshooting the OpenStack Image Service

Glance services

Confirming the Glance database

Confirming the Glance authentication

Keystone up

Service User Set Up

The service user setup

Service endpoints correct

Confirming the Glance API setup

Checking the command-line interface client

Glance logging and configuration

The log level

Where to look

Searching logs

Common errors

Unable to establish connection

Internal server errors (HTTP 500)

Unable to validate token

Summary

4. Troubleshooting OpenStack Networking

Identifying Neutron issues

Neutron services and agents

Neutron logs

Common problems

When you can't ping an instance

Security groups

Network namespaces

No IP address

Troubleshooting tools

ovs-vsctl

The Neutron client

Summary

5. Troubleshooting OpenStack Compute

Checking the services

nova-api

Address already in use

The permission error

nova-scheduler

nova-compute

nova-conductor

Supporting services

The Nova database

Nova authentication

Keystone up

Setting up the service user

Service endpoints correct

Nova and Glance

Nova and Neutron

Summary

6. Troubleshooting OpenStack Block Storage

Cinder processes

Logging

Cinder dependencies

Keystone authentication problems

RabbitMQ problems

Cinder errors

Missing the cinder-volumes volume group

The volume stuck in the creating state

Insufficient free space

Not sending heartbeat

Summary

7. Troubleshooting OpenStack Object Storage

Swift processes

The proxy server

Swift authentication

Troubleshooting TempAuth

TempAuth configuration

The account and username

The password

400 Bad Request

Troubleshooting Swauth

Swauth initialization

Swift with Keystone

Swift users

Summary

8. Troubleshooting the OpenStack the Orchestration Service

Heat services

Running heat-api

Running heat-engine

Heat authentication

The Keystone service

Auth credentials

Heat template errors

Summary

9. Troubleshooting the OpenStack Telemetry Service

Ceilometer processes

Ceilometer authentication

Ceilometer dependencies

The message broker

Databases

The command-line client

The meter list command

Glance meters

Nova meters

The sample list

Summary

10. OpenStack Performance, Availability, and Reliability

Databases

Availability

MySQL with Galera Cluster

Postgres

Performance

MySQL

Show status

Mytop

The Mytop header section

The Mytop thread section

Percona Toolkit

Postgres

The PostgreSQL statistics collector

Database backups

Monitoring

Resource monitoring

OpenStack quotas

RabbitMQ

Clustering

Mirrored queues

Services

Monitoring service processes

Backing up services

Community resources

Testing

Bugs

Ask.openstack.org

Summary

Index

Troubleshooting OpenStack

Copyright © 2016 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide tra...

Table of contents

• Troubleshooting OpenStack