Learning Core Audio

Chris Adamson is an independent writer, editor, and developer who lives in Grand Rapids, Michigan. Now focusing on iOS and Mac development, he is the coauthor of iOS SDK Development. He is also the author of QuickTime for Java: A Developer's Notebook and coauthor of Swing Hacks. He was formerly the editor of java.net and ONJava.com. He consults and publishes through his corporate identity, Subsequently and Furthermore, Inc., with a focus on user-facing and digital media development for Mac and iOS. He blogs on digital media software development at www.subfurther.com/blog. In a previous career, he was a writer/associate producer at CNN Headline News, and over the years, he has managed to own 11 1/2 Macs. Kevin Avila (a.k.a. dogbert) is a smooth blend of carbon compounds, oxygen, hydrogen, and nitrogen, with some impurities for added flavor. Additionally, he has more than 15 years' experience developing for the Mac and, since its release, the iPhone. Kevin has been involved in every corner of the audio market, from being an engineer at Apple to configuring professional recording studios. He currently is a code mercenary for various clients while he sits in his underwear at home, sipping coffee.

About the Authors     xiii Foreword  by Mike Lee  xv

Introduction     1

   Audience for This Book     2

   What You Need to Know     3

   Looking Up Documentation     3

   How This Book Is Organized     5

   About the Sample Code     9

 

PART I: UNDERSTANDING CORE AUDIO

Chapter 1  Overview of Core Audio     13

The Core Audio Frameworks     14

Core Audio Conventions     15

Your First Core Audio Application     16

   Running the Example     19

Core Audio Properties     22

Summary     23

 

Chapter 2  The Story of Sound     25

Making Waves     25

Digital Audio     27

DIY Samples     32

Buffers     40

Audio Formats     40

Summary     41

 

Chapter 3  Audio Processing with Core Audio     43

Audio Data Formats     43

   Example: Figuring Out Formats     46

Canonical Formats     51

Processing Audio with Audio Units     53

The Pull Model     55

Summary     55

 

PART II: BASIC AUDIO

Chapter 4  Recording     59

All About Audio Queues     59

Building a Recorder     60

A CheckError() Function     63

Creating and Using the Audio Queue     64

Utility Functions for the Audio Queue     71

   The Recording Audio Queue Callback     75

Summary     78

 

Chapter 5  Playback     81

Defining the Playback Application     81

Setting Up a File-Playing Audio Queue     83

   Setting Up the Playback Buffers     85

   Starting the Playback Queue     88

Playback Utility Functions     89

   Handling the Magic Cookie     89

   Calculating Buffer Size and Expected Packet Count     90

The Playback Audio Queue Callback     91

Features and Limits of Queue-Based Playback     94

Summary     95

 

Chapter 6  Conversion     97

The afconvert Utility     97

Using Audio Converter Services     100

Setting Up Files for Conversion     102

   Calling Audio Converter Services     105

   Implementing the Converter Callback     109

Converting with Extended Audio File Services     112

   Reading and Converting with Extended Audio Files     116

Summary     118

 

PART III: ADVANCED AUDIO

Chapter 7  Audio Units: Generators, Effects, and Rendering     123

Where the Magic Happens     123

How Audio Units Work     124

Sizing Up the Audio Units     126

Your First Audio Units     129

   Building the main() Function     131

   Creating an Audio Unit Graph     133

   Setting Up the File Player Audio Unit     137

Speech and Effects with Audio Units     141

   Building Blocks of the Speech Synthesis Graph     142

   Creating a Speech Synthesis AUGraph     144

   Setting Up a Speech Synthesizer     146

   Adding Effects     147

Adding Your Code to the Audio Rendering Process     150

   The Audio Unit Render Cycle     150

   A Custom Rendering Example     151

   Creating and Connecting Audio Units     154

   The Render Callback Function     155

Summary     160

 

Chapter 8  Audio Units: Input and Mixing     161

Working with I/O Input     161

   Connecting Input and Output Units     164

   Creating an AUHAL Unit for Input     168

   Writing the Input Callback     176

   Building an AUGraph to Play Samples from a CARingBuffer     178

   Writing the Play-Through App’s Render Callback     181

   Running the Play-Through Example     182

Mixing     183

Summary     189

 

Chapter 9  Positional Sound     191

Sound in Space     191

The OpenAL API     193

Putting a Sound in Space     196

   Setting Up the Example     197

   Using OpenAL Objects     200

   Animating the Source’s Position     205

   Loading Samples for an OpenAL Buffer     206

Streaming Audio in OpenAL     210

   Setting Up the OpenAL Streaming Example     210

   Setting Up an ExtAudioFile for Streaming     215

   Refilling the OpenAL Buffers     217

Summary     220

 

PART IV: ADDITIONAL TOPICS

Chapter 10  Core Audio on iOS     223

Is That Core Audio in Your Pocket?     223

Playing Nicely with Others: Audio Session Services     224

An Audio Session Example     227

   Setting Up the App     227

   Initializing the Audio Session and Audio Queue     231

   The Tone Generator Method     234 

   Handling iOS Interruptions     236

Audio Units on iOS     238

   Building an Audio Pass-Through App with the iOS RemoteIO Unit     239

   Setting Up the Pass-Through Example     241

   Setting Up the RemoteIO Audio Unit for Capture and Play-Out     244

   The RemoteIO Render Callback     249

Other iOS Audio Tricks     253

   Remote Control on iOS     253

   IOS Hardware Hazards     254

Summary     254

 

Chapter 11  Core MIDI     257

MIDI Concepts     257

Core MIDI     258

   Core MIDI Architecture     258

   Core MIDI Terminology     258

   Core MIDI Properties     260

   MIDI Messages     260

Instrument Units     261

Building a Simple MIDI Synthesizer     262

   Connecting to MIDI     265

Handling MIDI Notifications and Events     267 

   Playing Your AUGraph     269

Creating MIDI Events     269

   Setting Up the MIDIWifiSource Example     269

   Setting Up MIDI over Wi-Fi     271

   Sending MIDI Messages     273

   Setting Up Your Mac to Receive Wi-Fi MIDI Data     275

Summary: MIDI Mastery … but Mobility?     277

 

Chapter 12  Coda     279

Still More Core Audio     279

Next Steps     280

   Digital Signal Processing     280

Lion and iOS 5     281

   AUSampler     281

   Core Audio on iOS 5     285

The Core Audio Community     286

Summary: Sounds Good     287

 

Index     289