Working as an Android developer I have had to face an unconventional task – the client was interested in the ability of voice processing in real time, that is, on the fly.

During my student years, I was an amateur sound engineer and had the opportunity to experiment with audio processing and work with various audio editors on my PC. It's important to understand that your voice is a type of sound, similar to what a musical instrument produces. To process audio signals while developing your Android app, you need a tool that can fully support this task.

However, it's worth noting that studio equipment and mobile devices are entirely different things. If you're looking for modern software designed to work with audio data, you should be aware that such applications can be expensive and require the computing power of a desktop computer, which exceeds the capabilities of an average tablet or phone.

Unfortunately, I wasn't able to find a ready-made solution for working directly with the voice. However, I did come across some libraries and SDKs that work with audio data, which I would like to share with you. Since the Java programming language used in Android development is not ideal for working with audio content, you will mostly have to deal with JNI and C++ code.

SoundTouch Audio Processing Library.

• www.surina.net/soundtouch
• LGPL license.

The library is an Open Source project and isn't an SDK for any specific platform. The developers claim that their tool will work on Windows, Mac OS, Linux & other *nixes, Raspberry Pi, Android, Apple iOS.

The project is a C++ library with quite a limited functionality. The tools for application developers are limited to speeding up/slowing down the tempo and changing the sound pitch. In case of Android development, as noted above, you will have to deal with JNI – to implement C++ code into the project.

One more point is that the library only works with files. The original WAV-file is processed and is saved as a separate one. In order to hear the result of the implemented effects you have to play the newly created file. There is no real-time automatic sound optimization and sound processing on the fly.

Bongiovi DPS SDK.

• sdk.bongiovidps.com
• An exclusively paid license. However, the fee isn't stated on the site. The given phrase "Please contact us to discuss pricing" suggests quite flexible pricing. Without a license the SDK will work for 10 minutes in demo mode.

The product is a fully-fledged SDK with the ability to be used both in iOS and Android projects. It is a toolkit for those who want to create their own Audio Enhancer.

What is an Audio Enhancer? It is real time processing of the file being played – the application of the common audio effects like "On the street", "In the concert hall" and others, equalizer, dynamic loudness control and so on. To sum up, such tools aim at making the sound being played richer. Naturally, the usage of a quality acoustic system (or headphones) is expected.

Comparing with the tools I dealt with before in terms of sound processing, I would categorize this SDK as a DSP processor. Like a great number of DSP plugins for recently popular mp3 player WinAmp, for example. Now you have a chance to develop one of your own.

In case you have a desire to please the market of media players for portable devices with your product which will have its own sounding and various presets, this SDK will be quite of use to you.

However, it has no pitch-shifters, voice coders or any other sound (including voice) processors.

Essentia.

• essentia.upf.edu
• Affero GPLv3 license.

It's an open-Source C++ library for audio analysis. It's not a framework, rather a collection of algorithms wrapped in a cross-platform library compatible with Linux, Mac OS X, Windows, iOS and Android.

It allows you to thoroughly review the audio file and to retrieve plenty of specific information and characteristics, the purpose of half of which we can only guess. Among others, there are beat detection, BPM, rhythm onset detection, rhythm transform, danceability, dynamic complexity, audio segmentation, key, scale, predominant melody and pitch, chords, spectral descriptors (spectral peaks, complexity, contrast, dissonance), time-domain descriptors (duration, loudness and a bunch of others), statistical descriptors (from mean to kurtosis and skewness). Actually, it's exhaustive information about an audio file.

How to make use of it? I suppose all this information can be used by complex audio editors like Sony Sound Forge or Adobe Audition. A full-scale audio editor of such class on a mobile device is unlikely to be in high demand. It's much more customary to edit an audio sitting in front of a PC screen. The developers, by the way, suggest using the library in a bunch with Python. And in case you are developing your own audio editor, spectral analysis, for example, will prove itself useful when performing noise suppression operation.

If to speak about the usage of this audio processing library for Android development, you will also have to use JNI and manually implement C++ code into the project.

Superpowered Audio SDK.

• superpowered.com
• Superpowered SDK is free for developing apps distributed in app stores. In other cases, it is necessary to contact Superpowered team for details. Superpowered HLS streaming technology is not a part of the SDK and is not free.

It's a collection of C++ libraries for processing audio on the fly.

The SDK is a cross-platform (iOS, Android and OSX) and is promoted by developers as the fastest tool for audio processing. Moreover, it is said to have reduced CPU load which prolongs battery life and some other pleasant features.

The developer is offered to apply different audio effects like reverberation, echo, compressor, limiter, flanger, gate, mixer. To sum up, the effects so familiar to people who are keen on playing the guitar and have tried audio editing. There are also tools for analyzing the source of the audio which can also be partially used for voice processing. The SDK pack includes documentation and application samples.

As for creating a project for Android, according to the developers, there is no full integration with an Android project yet. You will have to apply some magic when working with the Gradle project build system. However, this process is explained in the documentation.

SpectrumWorx SDK.

I would also like to recall a tool called SpectrumWorx SDK.

• littleendian.com/developers
• This SDK is paid.

There are demo-recordings with various audio effects open to public inspection. You can check out how everything will sound right away.

Trial versions of the SDKs are available for developers. In order to get one, you need to fill in a form and send a request. Unfortunately, I haven't received any reply to my request.

Finally.

Above, I mentioned the main tools for realtime audio processing that I have come across while searching for a solution. The list, obviously, can be updated and extended. I described only those I got acquainted with during my research. I sincerely hope this information will come in handy to you, my dear readers.

Thank you for your attention to this article.