Audio Visualization Using ThMAD (eBook)

Dr. Peter Späth has worked as a IT consultant with heavy focus on Java related development for over 15 years. Recently, Peter has decided to focus on his work has an author and working in a self-paced manner on software.

1. IntroductionSound visualization is about capturing the sound coming from outside, using a microphone or the line-in jack of your sound card, or coming from inside if a sound file gets played on your computer. The software suite ThMAD allows for graphically building a sound pipeline, yielding realtime graphics rendering to produce beautiful and interesting output from that sound. The introductory chapter depicts such kind of a system, tells about the targeted audience, and gives some hints about text conventions and the preferred way of reading this book.1.1 About Sound Visualization and ThMAD1.2 Targeted Audience1.3 Installation1.4 Conventions Used in This Text1.5 How to Read This Book2. Sound InputWe deepen the knowledge of how the computer can be prepared to capture incoming sound or produced sound, how the sound gets represented internally, and how the data arrives at ThMAD. We distinguish between the obvious air pressure elongation versus time representation, and the power versus frequency representation or spectrum.2.1 Prepare for Sound Input2.2 Understanding Sound Structure2.2.1 Time-Elongation Representation2.2.2 Frequency-Power Representation2.2.3 Input Data Taken From the Sound Card3. Visualization BasicsThis chapter is about the tool chain to be used for an audio visualization project, and presents two basic audio visualization samples using ThMAD.3.1 Tool Chain3.1.1 Installing ThMAD3.1.2 Connect PulseAudio Sound to ThMAD3.1.3 Recording a Video3.1.4 Making a DVD From Your Recording3.2 Basic Samples3.2.1 Basic 2D Sample3.2.2 Basic 3D Sample4. Program OperationThe ThMAD primarily consists of two programs, ThMAD Artiste and ThMAD Player. We describe how these programs can be started and stopped, including all possible options for controlling program operation. We furthermore describe faders, which conduct the transitions between several visualizations inside Player.4.1 ThMAD Artiste Operation4.1.1 Starting And Using Different Modes4.1.2 Stopping ThMAD Artiste4.1.3 Starting With Errors4.2 ThMAD Player Operation4.2.1 Starting And Using Different Modes4.2.2 Stopping ThMAD Player4.2.3 Creating and Installing Faders5. 3D ConceptsFor an exhaustive reference to 3D computer graphics concepts lots of materials are available in the internet and book stores. Albeit OpenGL, the technology behind ThMAD, being huge as a concept and a complete coverage of all topics exceeding the scope of this book, we reckon an introduction with our special view on audio visualization should be worthwhile. In addition some of the peculiarities about how ThMAD handles some graphics sub-pipelines and own graphics concepts on top of OpenGL get described later in this chapter.5.1 Coordinate Systems5.2 Space Mapping5.3 Spatial Operations: Translation, Rotation, Scaling5.4 Exposure to Light5.5 Eye And Camera5.6 ThMAD Meshes5.7 ThMAD Particle Systems6. StoriesIn this chapter we want to dive more into the art of audio visualization by more elaborated walk-throughs covering various aspects to be taken into account. We will by no means ever be able to describe all possible kinds of sketches, but it will be the aim of this Stories chapter to provide you with techniques which eventually will be apt to make you realize your own ideas. There is no particular order for the sections, so you can sequentially work through them or cross read them, whatever best suits you.6.1 More 3D Rendering Pipelines6.1.1 Transformations6.1.2 Wireframes6.1.3 Ocean6.2 Texture Mapping6.2.1 Automatic Texture Coordinates6.2.2 Floating Textures I6.2.3 Floating Textures II6.3 Backfeeding Textures6.3.1 Blurring in Two Dimensions6.3.2 Self-Similarity6.4 Blobs, Blobs, Blobs6.4.1 Basic Blobs6.4.2 Perlin Noise Blobs6.5 Particle Systems 6.5.1 Waterfall6.5.2 Image Bit Particles6.5.3 Center Clamped Particle Systems>6.5.4 Ribbon Particles6.6 Glowing Objects7. ThMAD Artiste GUI ReferenceThe subject of this chapter is an exhaustive manual of ThMAD Artiste GUI operations. All front-end elements get described, including the main menu, module choosers, some helper widgets, namely the Assistant and the Inspector, as well as different operation modes, and keyboard shortcuts. This is followed by the concepts of saving and loading of States which describe a visualization pipeline. We talk about what Modules are, what types of modules we have, how they can be controlled, and how they can be connected to other modules. The export of states for the purpose of using them later from inside ThMAD Player, the description of macros for defining sub-pipelines, and the introduction of Note widgets complete this chapter.7.1 Starting and Stopping the GUI7.2 The ThMAD Desktop And Its Parts7.3 Window modes 7.3.1 Fullwindow Mode7.3.2 Performance Mode7.3.3 Fullscreen Mode7.4 The Main Menu7.5 Module Choosers7.5.1 The Module List7.5.2 The Graphical Module Chooser7.6 The Assistant7.7 The Object Inspector7.8 Saving and Loading States7.9 Modules7.9.1 Module Types7.9.2 Placing and Deleting Modules7.9.3 Connecting Modules7.9.4 Cloning Modules7.10 Module Anchors: Parameters And Connectors7.10.1 Drawing Connections Between Anchors7.10.2 Enumeration Input as Module Parameter 7.10.3 Float Input as Module Parameter7.10.4 Float3 Input as Module Parameter7.10.5 Float4 Input as Module Parameter7.10.6 Quaternion Input as Module Parameter7.10.7 String Input as Module Parameter7.10.8 Resource as Module Parameter7.10.9 Sequence Input as Module Parameter7.11 Exporting States7.12 Macros7.13 Notes8. ThMAD Player GUI ReferenceThMAD player is the part of the ThMAD program suite responsible for presenting finished states, which inside the Player’s nomenclature are called Visualizations. Inside this chapter we describe all GUI operation options, more precisely the keyboard shortcuts which can be used to control ThMAD Player.8.1 Starting and Stopping the GUI8.2 Player GUI Operations9. ThMAD Module ReferenceHere we provide a complete module reference, both for built-in modules and plug-in modules. By its structure it gives you an oversight of what can be done using the software, and both scanning over the chapter and reading it thoroughly will help you get some impressions and new ideas. The module category names and the module names themselves closely relate to the GUI and the implementation code, with an emphasizing to the GUI so you can easily make your own experiments using the GUI.9.1 Screen9.1.1 screen09.2 Bitmaps9.2.1 Filters9.2.2 Generators9.2.3 Loaders9.2.4 Modifiers9.3 Dummies9.4 Maths9.4.1 Accumulators9.4.2 Arithmetics9.4.3 Array9.4.4 Color9.4.5 Converters9.4.6 Dummies9.4.7 Interpolation9.4.8 Limiters9.4.9 Oscillators9.5 Mesh9.5.1 Dummies9.5.2 Generators9.5.3 Importers9.5.4 Modifiers – Color9.5.5 Modifiers – Converters9.5.6 Modifiers – Deformers9.5.7 Modifiers – Helpers9.5.8 Modifiers – Pickers9.5.9 Modifiers – Transforms9.5.10 Particles9.5.11 Segmesh9.5.12 Solid9.5.13 Texture9.5.14 Vertices9.5.15 Xtra9.6 Particlesystems9.6.1 Fractals9.6.2 Generators9.6.3 Modifiers9.7 Renderers9.7.1 Basic9.7.2 Mesh9.7.3 OpenGL Modifiers9.7.4 Oscilloscopes9.7.5 Particlesystems9.7.6 Shaders9.7.7 Text9.7.8 Xtra9.8 Selectors9.9 Sound9.9.1 input_visualization_listener9.9.2 midi → aka_apc40_controller9.9.3 ogg_sample_*9.9.4 raw_sample_*9.10 String9.11 System9.12 Texture9.12.1 Buffers9.12.2 Dummies9.12.3 Effects9.12.4 Loaders9.12.5 Modifiers9.12.6 OpenGL9.12.7 Particles10. Appendix10.1 Figures