Algorithms

Sound Event Detection

1. YamNet, and Real-time sound event detection github, Event types labels list - Relevant labels: 420:430

Query-based separation

1. Zero Shot Audio Source Separation, paper, interface - is a three-component pipeline that allows you to train an audio source separator to separate any source from the track. All you need is a mixture audio to separate, and a given source sample as a query. Then the model will separate your specified source from the track.

Audio Source Separation

1. Audio Sep - AudioSep is a foundation model for open-domain sound separation with natural language queries. AudioSep demonstrates strong separation performance and impressive zero-shot generalization ability on numerous tasks such as audio event separation, musical instrument separation, and speech enhancement"

2. Wave-U-net

   1. Original Version 4y old

   2. Pytorch Version 3y old

   3. TF2 / Keras Version 2y old

   4. For speech enhancements

Blind Source Separation

1. Deep Audio Prior - Our deep audio prior can enable several audio applications: blind sound source separation, interactive mask-based editing, audio textual synthesis, and audio watermarker removal.

2. BSS (EM source separation) - This repository covers EM algorithms to separate speech sources in multi-channel recordings. In particular, the repository contains methods to integrate Deep Clustering (a neural network-based source separation algorithm) with a probabilistic spatial mixture model as proposed in the Interspeech paper "Tight integration of spatial and spectral features for BSS with Deep Clustering embeddings" presented at Interspeech 2017 in Stockholm.

Image embeddings and others

1. Openl3 - OpenL3: Open-source deep audio and image embeddings

2. Pitch estimation

3. Speaker recognition - Speaker recognition is the identification of a person given an audio file. It is used to answer the question "Who is speaking?" Speaker verification (also called speaker authentication) is similar to speaker recognition, but instead of returning the speaker who is speaking, it returns whether the speaker (who is claiming to be a certain one) is truthful or not. Speaker Verification is considered to be a little easier than speaker recognition.

4. Voice activity detector

5. Taken from here

Other Tools

1. KALDI speech recognition toolkit with many SOTA models.

2. isolating instruments from stereo music using Convolutional Neural Networks, part 2

3. Sound classification using cnn, loading and normalizing sounds using librosa, converting to a 2d spectrogram image, using cnn on top.

4. speech recognition with DL - how to convert sounds to vectors, feeding into an RNN.

5. (Great) Jonathan Hui on speech recognition - series.

6. Gecko - (github.com/gong-io/gecko) youtube, is an open-source tool for the annotation of the linguistic content of conversations. It can be used for segmentation, diarization, and transcription. With Gecko, you can create and perfect audio-based datasets, compare the results of multiple models simultaneously, and highlight differences between transcriptions.