Deep Learning and IoT Framework Records, Preserves and Recreates Traditional Music

A new study proposes a framework that combines Internet of Things (IoT) sensors with deep learning to record, analyse, preserve and creatively regenerate traditional music. Published in Discover Internet of Things, the research by Tianbo Guo describes a system in which IoT-enabled sensors capture real-time performances of vocal, instrumental and dramatic music, logging acoustic, temporal and environmental parameters such as tempo, duration, acoustic level, frequency centroid, pitch variability, rhythm complexity and noise level. The work addresses a practical concern raised in the study: rapid digitalisation and a lack of systematic preservation threaten the transmission of invaluable musical resources.

The underlying dataset, called the Digital Inheritance and Innovations in Music Dataset, contains 2,905 recordings of traditional, contemporary and modern music gathered with IoT-based sensors, sourced from a publicly available Kaggle collection. Alongside numeric variables, the data include categorical labels such as music type, emotion tags and recording environment. For the experiments, 80% of the data trained the models and the remaining 20% was reserved for testing. Hardware consistency was addressed through a calibration procedure in which all IoT audio sensors were checked against a reference microphone with identical sample rate, bit depth and frequency range settings, and devices were verified by signal-to-noise ratio before use.

Before any learning takes place, the raw audio passes through a three-stage preprocessing pipeline. A Wiener filter, implemented in the frequency domain, suppresses noise by balancing noise smoothing against inverse filtering and minimising the mean square error between desired and observed signals. Z-score normalization then standardises features such as tempo and frequency to a common scale with zero mean and unit standard deviation, reducing bias across different recording devices. Finally, the Short-Time Fourier Transform (STFT) converts the non-stationary audio into a time-frequency spectrogram using a window size of 1024, yielding 513 frequency bins. From these spectrograms the team extracted mel-frequency cepstral coefficients for timbre, chroma features for harmonic information, spectral centroid, spectral bandwidth, zero-crossing rate and tempo.

The core recognition model is a Versatile Deep Neural Network (VDNN), a fully connected feedforward network with an input layer, two hidden layers of 128 and 64 neurons and a softmax output layer. ReLU activations supply non-linearity in the hidden layers, and softmax cross-entropy loss drives training. Because raw acoustic signals can produce redundant or overlapping features, the network’s structure is modified using a Gaussian mixture model, which extracts statistical characteristics of the sounds, reduces feature dimensionality and shrinks the model. Notably, this design choice is borrowed from underwater acoustic target recognition, where directly feeding raw signals leads to duplicated spectral features.

Training alone, however, was not the whole story. The authors wrapped the VDNN in an Intelligent Water Drop (IWD) optimisation algorithm, a nature-inspired metaheuristic modelled on how flowing water carves paths of least resistance. In this implementation, artificial water drops traverse a graph of candidate solutions, updating “soil” values on edges so that parameter combinations producing smaller prediction errors become more likely to be selected. The IWD handles weight initialisation and hyperparameter tuning, and the resulting parameter set then seeds standard gradient-based learning with backpropagation. The network was trained with the Adam optimizer at a learning rate of 0.001, a batch size of 32, 100 epochs, dropout of 0.5 and L2 weight decay of 1 × 10⁻4.

For the creative side of the framework, the researchers added a generative module built on an encoder-decoder architecture. The encoder compresses STFT-derived acoustic features into a latent space representing rhythmic, tonal and stylistic characteristics, with a variational autoencoder enabling interpolation between musical sequences. The decoder is a hybrid Transformer-LSTM model that generates new sequences by learning long-range temporal relationships, while probabilistic sampling with temperature tuning introduces controlled variation so that outputs remain stylistically coherent.

The reported results are striking, though they come from the authors’ own benchmark comparisons. The IWD-VDNN achieved 94% accuracy, 84.2% precision, 92.2% recall and a 93.6% F1-score on the music dataset. Against the baselines trained on identical features and preprocessing, CNN reached 85.7% accuracy, RNN 80.5%, SVM 75.2% and an Improved Genetic Chord Algorithm 92.5%. On generation quality, the Musical Quality Score, assessed by professional musicians and musicologists on a five-point Likert scale covering harmony, melodic continuity, rhythmic consistency, timbral authenticity and aesthetic quality, rose from 4.68 to 4.91 relative to the chord-algorithm baseline. Style Match Accuracy improved from 93 to 95%, Style Consistency from 95 to 97%, and generation time fell from 0.15 to 0.11 seconds. A stratified five-fold cross-validation, with preprocessing repeated inside each training fold to prevent data leakage, reported average accuracy of 94.2% and recall of 92.2%.

The framework also showed measurable gains in preservation-oriented metrics when the IWD optimiser was added to the base VDNN. Inheritance rates for vocal music rose from 76 to 78%, instrumental from 93 to 95% and drama music from 88 to 90%. Pattern fidelity improved from 84 to 86%, temporal accuracy from 82 to 84%, timbre preservation from 80 to 82% and overall retention from 85 to 87%. On the innovation side, creativity scores moved from 78 to 80%, novelty from 81 to 83%, and audience satisfaction from 83 to 85%. Exploratory visual analyses of the dataset further suggested that vocal music showed uniformly high rhythmic variation while instrumental music was more stable, and that emotional categories such as excitement and anger displayed wider rhythmic distributions.

The authors are candid about the framework’s weaknesses. The VDNN on its own carries high computational cost, tends to overfit smaller datasets, is sensitive to noise and offers poor interpretability of learned musical features. Accuracy also depends heavily on IoT hardware quality, since differences in microphone sensitivity and frequency characteristics can introduce artifacts that complicate extraction of subtle attributes like timbre and pitch. Generalisation across cultures remains a major open question: tonal systems with microtones, differing rhythmic conventions and underrepresented musical traditions may all degrade performance, and the authors call for cross-cultural validation, larger and more diverse datasets, transfer learning and domain adaptation in future work.

Whether such systems can faithfully honour the microtonal nuance and expressive depth of the world’s less-documented musical cultures is the test the authors themselves set for the next round of research.

Subject of Research: A deep learning and Internet of Things framework for digitally preserving and creatively regenerating traditional music.

Article Title: Digital inheritance and innovative development of music based on deep learning and the Internet of Things

Article References: Guo, T. (2026). Digital inheritance and innovative development of music based on deep learning and the Internet of Things. Discover Internet of Things, 6(1), Article 204. https://doi.org/10.1007/s43926-026-00510-8

Image Credits: AI Generated

DOI: 10.1007/s43926-026-00510-8

Keywords: deep learning, Internet of Things, traditional music, digital heritage, music generation, Intelligent Water Drop algorithm, neural network, signal processing, cultural preservation, machine learning, Digital, inheritance