Stochastic Optimization for Large-scale Machine Learning

Vinod Kumar Chauhan

• Publisher: CRC Press
• Year: 2021
• Print ISBN:
  9781032146140
  9781032131757
• Edition: 1
• eBook ISBN:
  9781000505610
• Topic: Computer Science
• Subtopic: Operations
• Index: Computer Science

Advancements in the technology and availability of data sources have led to the `Big Data' era. Working with large data offers the potential to uncover more fine-grained patterns and take timely and accurate decisions, but it also creates a lot of challenges such as slow training and scalability of machine learning models. One of the major challenges in machine learning is to develop efficient and scalable learning algorithms, i.e., optimization techniques to solve large scale learning problems.

Stochastic Optimization for Large-scale Machine Learning identifies different areas of improvement and recent research directions to tackle the challenge. Developed optimisation techniques are also explored to improve machine learning algorithms based on data access and on first and second order optimisation methods.

Key Features:

• Bridges machine learning and Optimisation.
• Bridges theory and practice in machine learning.
• Identifies key research areas and recent research directions to solve large-scale machine learning problems.
• Develops optimisation techniques to improve machine learning algorithms for big data problems.

The book will be a valuable reference to practitioners and researchers as well as students in the field of machine learning.

II_________FIRST ORDER METHODS

Chapter 3 Mini-batch and Block-coordinate Approach

DOI: 10.1201/9781003240167-3

Big Data problems in Machine Learning have large number of data points or large number of features, or both, which make training of models difficult because of high computational complexities of single iteration of learning algorithms. To solve such learning problems, Stochastic Approximation offers an optimization approach to make complexity of each iteration independent of number of data points by taking only one data point or mini-batch of data points during each iteration and thereby helping to solve problems with large number of data points. Similarly, Coordinate Descent offers another optimization approach to make iteration complexity independent of the number of features/coordinates/variables by taking only one feature or block of features, instead of all, during an iteration and thereby helping to solve problems with large number of features. In this chapter, an optimization framework, namely, Batch Block Optimization Framework has been developed to so...

Table of contents

• Cover Page
• Half-Title Page
• Title Page
• Copyright Page
• Dedication Page
• Contents
• List of Figures
• List of Tables
• Preface
• SECTION I BACKGROUND
• SECTION II FIRST ORDER METHODS
• SECTION III SECOND ORDER METHODS
• SECTION IV CONCLUSION
• Bibliography
• Index